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(54) Efficient and secure update of software and data 



(57) The invention concerns apparatus for updating 
a data file, from an earlier version to a later version. The 
invention compares the earlier version with a later ver- 
sion, and derives a transformation, which contains infor- 
mation as to the similarities and differences. The 
invention then processes the earlier version, using the 
transformation, in order to derive the later version, with- 
out reference to the later version itself. 



The invention allows multiple versions of an original 
file, such as bank records located at multiple locations, 
to be updated by transmitting the transformation to the 
multiple locations, instead of transmitting the updated 
files themselves. The procedure is highly resistant to 
interception of the updated data, because the transfor- 
mation, in general, does not contain the entire contents 
of the later versions. 
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Description 

An APPENDIX is attached which contains source code used by the invention. The invention concerns a method for 
remotely updating software or data, or both, which reduces the amount of data transmitted, and is less susceptible to 
5 interception. 

Background of The Invention 

Electronic data transmission between computers via a common-carrier telephone system requires speed and secu- 
re? rity. Transmission speed can be enhanced by compressing the data before transmission, while security can be protected 
using encryption. Data compression is also beneficial for encryption because it reduces redundancy in the data source, 
and increases resistance to cracking attempts. 

However, if a data f Be is compressed and encrypted, the resulting compressed-and-encrypted data is based on a 
single source. Data compression and encryption based on single data sources are not perfect Unless great redundancy 
15 exists in the data, compression does not perform well, and high-speed computers working together with modern crypto- 
analytic methods can crack most encrypted data. 

In many situations, multiple later versions of an initial data source are frequently transmitted after slight changes in 
the data. The fact that the changes are slight means that all versions are inherently similar. The inherent similarity can 
be exploited to compute a minimal transformation that transforms one version into another. 
20 This process of computing such a transformation is called data differencing. Data differencing enhances transmis- 
sion speed because it reduces the amount of data to be communicated. It also enhances privacy because a snooper 
without the earlier version cannot deduce too much information from the transmitted data 

A number of compression and differencing techniques have been previously considered. The article "A Universal 
Algorithm for Sequential Data Compression," by J. Ziv and A. Lempel, contained in IEEE Transactions on Information 
25 Theory, 23(3): pp. 337 - 343, May, 1977, describes a technique to compress a single data file. This works by analyzing 
a data sequence and, at each position, if possible, identifying the longest segment that matches another segment in the 
already analyzed part Such a segment, if found, is encoded using the matching position and the length of the match. 
Unmatched data are output as-is. 

An implementation of this algorithm was the subject of U.S. Patent number 4,464,650, August 7, 1984, by W.L 
30 Eastman, A. Lempel, and J. Ziv. The Eastman-Lempel-Ziv compression method is slow on de-compression because it 
has to do roughly the same amount of work during de-compression as during compression. It also does not apply to 
data differencing. 

On UNIX systems, differences between text (not binary) files can be computed using the program "diff," which 
produces a set of lines that must be deleted or added to transform the first file to the second. The "diff " approach can 

35 produce very large transformations because small changes in text lines cause them to be considered different. Further, 
it only works on text files, so it has a limited range of applicability. 

The article "The String-to-String Correction Problem with Block Moves," by Walter F. Tichy, contained in ACM Trans- 
actions on Computer Systems, 2(4): pp. 309 - 321 , November, 1 984, describes an algorithm to compute a set of instruc- 
tions called block-moves to transform one version of a data file to another. A block-move is a data segment in the second 

40 version that matches another segment in the first version. This algorithm is essentially unrealized and does not take 
advantage of any redundancy inherent in the second version to minimize the transformation needed to construct the 
second version from the first version. 

Summary of The Invention 

45 

In one form of the invention, a computer program compares a first and a second, later, version of a file, and produces 
a sequence of instructions. The instructions allow replication of the second version, based on the first version. 

The instructions are of two types: One type is a COPY instruction, which orders copying of a specified sequence of 
characters into a file being constructed. The specified sequence can reside either in the first version, or in the file under 
so construction. 

The second type is an ADD instruction, which orders addition of a sequence of characters, which accompanies the 
ADD instruction. When the instructions are executed in sequence, they produce a concatenation of sequences, which 
are COPIED and ADDED, producing a replica of the second version. 

55 Brief Description of The Drawings 

Figure 1 is a schematic of how the invention is used for data communication between computers. 
Figures 1 A through 1 1 explain, by analogy, principles used by the invention. 
Figure 1J illustrates a security aspect of the invention. 
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Figure 2 gives an example of two versions of a data set and a set of instructions to transform one of the versions to 
the other. 

Figures 2A through 2E illustrate operation of the procedure described in Figure 3. 

Figures 2F through 2H illustrate operation of the procedure descrfoed in Figure 5. 
5 Figure 3 describes the decoding procedure in a pseudo-C language. 

Figure 4 shows the actual encoding of the instructions of Figure 2. 

Figure 5 describes the encoding procedure. 

Figure 6 describes the INSERT procedure used in Figure 5. 

Figure 7 describes the SEARCH procedure used in Figure 5. 
10 Figure 8 describes the EXTEND procedure used in Figure 5. 

Figure 9 shows the inserted positions of the example VERSION1 and VERSION2 data of Figure 2. 

Figure 10 shows the secure version of the coded instructions of Figure 4. 

Detailed Description of The Invention 

15 

A simplified analogy will explain some of the more fundamental aspects of the invention. A technical explanation of 
computer code which implements part of the invention is given later, beginning in the section entitled The Invention". 

Analogy 

20 

Assume that a newspaper reporter and a newspaper editor are writing an article together, but at different locations. 
Assume that they both have written VERSION 1 , which is shown in Figure 1 A. Assume that the reporter has made the 
changes shown in Figure 1B, resulting in VERSION 2, which is shown in Figure 1C (the boxes indicate the locations of 
changes), and wishes to send this version to the editor. The following procedure will allow the reporter to send VERSION 
25 2 to the editor, but without sending the entire contents of VERSION 2. 

Procedure 

First, the reporter assigns a position to each word in VERSION 1 , as indicated in Figure 1 A. There are 50 positions, 
30 corresponding to the 50 words. 

Second, the reporter ascertains the vocabulary of VERSION 1 . The vocabulary is a list, or table, of the words used, 
as indicated in Figure 1 A. Note that each word appears in the vocabulary only once. For example, the word "them" at 
position 18 is not entered into the vocabulary, because this word previously appeared, at position 16. 

In order to assure that each word appears only once, the reporter, as each word is entered, checks to see whether 
35 that word already exists in the table. For example, the reporter enters the first word, "when." Then, the reporter takes 
the word at position 2, "buying," and checks it against the first word in the vocabulary, namely, "when". There is no match, 
so the reporter enters "buying" into position 2 in the vocabulary. 

The reporter proceeds in this manner until the word "them" is reached at position 18. When checking this word 
against the previous 17 entries in the vocabulary, the reporter finds that "them" matches "them" at position 16 in the 
40 vocabulary. Consequently the latter "them" is not entered into the vocabulary, because it would duplicate the previous 
"them." 

When the last word in VERSION 1 is checked for matches, the vocabulary becomes complete. VERSION 1 , which 
is 50 words in overall length, contains only 38 different words, as indicated by the length of the vocabulary. 

Next, the reporter establishes a vocabulary for VERSION 2, as shown in Figure 1C. VERSION 2 has the same 
45 vocabulary as VERSION 1 , with the exception of three additional words, indicated as NEW WORDS. Now the reporter 
is ready to transmit VERSION 2 to the editor. 

The reporter transmits VERSION 2 by transmitting the following six messages, or instructions, each of which will 
be explained by a respective Figure in Figures 1 D - 11: 

so 1. COPY 2 1 

2. ADD 1 Hawaiian 

3. COPY 3 3 

4. ADD 1 all 

5. COPY 37 8 

55 6. ADD 2 a brick. 
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Instruction 1 

By pre-arrangement, the reporter and the editor set a POINTER at position 1, as indicated in Figure 1D. The first 
instruction, "COPY 2 1" means "COPY the sequence of 2 words which begin at position 1 in VERSION 1." (In each 
5 instruction, it is implicit that the words are placed at the POINTER in VERSION 2.) This operation is shown in Figure 1 D. 
Note that the syntax is 

COPY [number of words, starting position in VERSION 1]. 
After following the instruction, the editor moves the POINTER by the "number of words" in the COPY instruction, 
which, in this case, is 2. The POINTER is now located as shown in Figure 1 E. 

10 

Instruction 2 

instruction 2 states "ADD 1 Hawaiian," which means "ADD the one word 'Hawaiian\" This operation is shown in 
Figur 1 E. Note that the syntax is 
15 ADD [number of words, words themselves]. 

After following the instruction, the editor moves the POINTER by the "number of words" in the ADD instruction, 
which, in this case, is 1 . The POINTER is now located as shown in Figure 1 F. 

Instruction 3 

20 

The third instruction, "COPY 3 3" means "COPY the sequence of 3 words which begin at position 3 in VERSION 
1 This operation is shown in Figure 1 F. Note that the syntax is identical to that of instruction 1 . 

After following the instruction, the editor moves the POINTER by the "number of words" in the copy instruction, 
which, in this case, is 3. The POINTER is now located as shown in Figure 1G. 

25 

Instruction 4 

Instruction 4 states "ADD 1 all." which means "ADD the one word 'all'." This operation is shown in Figure 1G. 
After following the instruction, the editor moves the POINTER by the "number of words" in the ADD instruction, 
30 which, in this case, is 1 . The POINTER is now located as shown in Figure 1 H. 

Instruction 5 

The fifth instruction, "COPY 37 8" means "COPY the sequence of 37 words which begin at position 8 in VERSION 
35 1 This operation is shown in Figure 1 H. Note that the syntax is identical to that of instruction 1 . 

After following the instruction, the editor moves the POINTER by the "number of words" in the copy instruction, 
which, in this case, is 37. The POINTER is now located as shown in Figure 1 1. 

Instruction 6 

40 

Instruction 4 states "ADD 2 a brick," which means "ADD the 2 words a brick ." This operation is shown in Figure 1 1. 
Now VERSION 2 has been constructed. 

Significant Features 

45 

VERSION 2, shown in Figure 1C, contained 43 words, which themselves contained 163 characters. However, the 
set of six instructions contained 27 characters, plus the instructions themselves (ie, COPY and ADD). If each instruction 
and each character is encoded as one byte, then the total message equals 27 + 6 characters, or 33 characters. Trans- 
mitting 33 characters takes a significantly shorter time than transmitting 163 characters. 

50 

The Invention 

The previous analogy was a simplification, and does not illustrate all of the features of the invention. Figure 2 illus- 
trates two VERSIONS of a file, together with a TRANSFORMATION, which allows construction of VERSION2 from the 
55 combination of VERSION1 and the TRANSFORMATION itself. The procedure shown in Figure 3 accomplishes the 
reconstruction, and will now be explained. First, the general pattern will be elaborated. 

As background, four important points should be recognized. One, each character in each VERSION occupies a 
numbered position, as shown in Figure 2A. For example, in VERSION 1 , the first "a" occupies position 0. The first "b" 
occupies position 1 , and so on. 
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Two, the numbered positions in VERSION 2 begin after the highest-numbered position in VERSION 1. Thus, since 
the highest position in VERSION 1 is "15," VERSION 2 begins with "16." 

Three, a pointer (which is a variable in the procedure of Figure 3) indicates a current position, as indicated in Figure 

2A. 

5 Four, the TRANSFORMATION contains instructions. There are two types of instruction, namely, ADD and COPY. 

The operation of these two instructions will be explained by showing how VERSION 2 is constructed from VERSION 1 . 

Instruction 1 

w In Figure 2B, instruction 1 of the TRANSFORMATION is executed. This instruction, "COPY 4 0," states, in effect. 
" COPY the FOUR characters beginning at position ZERO in VERSION 1 to the pointer in VERSION 2." (The underlined 
words refer to the words in the instruction: FOUR refers to 4, and so on.) Figure 2B illustrates the operation. 

It is significant that; in this instruction, the characters used to generate VERSION 2 are obtained from VERSION 
1 , not from the instruction itself. Thus, if the instruction were obtained via a telephone transmission, an eavesdropper 
75 would obtain no information usable in constructing VERSION 2 because the eavesdropper is presumed to have no 
access to VERSION 1. 
The syntax is 

COPY [number of characters, starting position] 

20 Instruction 2 

In Figure 2C, instruction 2 is executed. This instruction, "ADD 2 x.y," states, in effect " ADD the TWO characters x 
and y to VERSION 2, beginning at the pointer." v 
It is significant that, in this instruction, the characters used to generate VERSION 2 are obtained from the instruction 
25 itself. Consequently, an eavesdropper does obtain some information about VERSION 2. However, in practice, this type 
of instruction is expected to be intermixed with so many types of COPY instructions (which contain no information) that 
the eavesdropper would obtain no significant information about VERSION 2. 

Nevertheless, it is possible, in theory, that the TRANSFORMATION may contain ADD instructions exclusively, in 
which case the eavesdropper would obtain VERSION 2 in its entirety. Or the number of ADD instructions may be very 
30 large. To prevent an eavesdropper from obtaining information from these ADD instructions, an encryption option is pro- 
vided, which is discussed later. 

The syntax is of the ADD instruction is 
ADD [number of characters, characters themselves] 

35 Instruction 3 

In Figure 2D, instruction 3 is executed. This instruction, "COPY 6 20", in effect states: 
generate a COPY which is six characters long, using the characters residing between position TWENTY and the 
pointer, and place them at the pointer. 
40 The result is the sequence xyxyxy, as indicated. 

(As another example, if the COPY statement read "COPY 6 19", then it, in effect, would have stated 

generate a COPY, which is six characters long, using the characters residing between position NINETEEN (not 
TWENTY) and the pointer, and place them at the pointer. 

The sequence added at the pointer would have been dxydxy, because the sequence residing between position 
45 nineteen and the pointer is dxy.) 

It is significant that, in this instruction, the characters used to generate VERSION 2 are obtained from VERSION 2 
itself. Thus, it is now seen that the COPY instruction uses two sources, depending on the address specified. In this 
instruction, the address (ie, the "20" in "COPY 6 20") indicates VERSION 2 (ie, the address is greater than 1 5). VE RSION 
2 is the source for this command. 
so In contrast, if the address were 15 or less, then VERSION 1 would have been used as the source. 

This approach, in effect, allows one to expand the vocabulary, using ADD instructions. Then, when VERSION 2 is 
found to contain characters which are (a) not in VERSION 1, but (b) contained in the expanded vocabulary (because 
added earlier), a COPY instruction can be used. The COPY instruction has two benefits. 

One is that a COPY instruction can insert a large number of characters, merely by indicating their total number, and 
55 starting position. In contrast, the ADD instruction requires the characters themselves to be transmitted, which entails a 
longer transmission. 

A second is that, as discussed above, the COPY instruction contains no information, unless an interceptor of the 
instruction also happens to possess VERSION 1. 
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Instruction 4 

In Figure 2E, instruction 4 is executed. This instruction, "COPY 5 9," in effect, states: "COPY the FIVE characters 
beginning at position NINE in VERSION 1 This instruction resembles instruction 1. 

5 

Reference to Process of Figure 3 

Figure 3 illustrates a process by which a computer can construct VERSION 2 from the combination of VERSION 1 
and the TRANSFORMATION. 

10 In Figure 3, line 2 initializes a variable c, which is used to compute the pointer shown in Figures 2B - 2E. The pointer 
is computed in lines 9,, 13, and 14. as appropriate. 

Instruction 1 

15 Instruction 1 (COPY 4 0) of the TRANSFORMATION shown in Figure 2 is executed in line 13 in Figure 3. The 
variabl "p" indicates the starting position of the source data, is obtained from the instruction, and equals 0. Since "p" 
is less than "n" (line 12), the source lies in VERSION 1 . Consequently, line 13 is executed, because of the IF-statement, 
and the data is copied from VERSION 1, beginning at position VERSION1+p, which is position 0. The variable "s" 
indicates the number of characters, is obtained from the instruction in line 7, and equals 4. 

so The pointer is updated on line 16. 

Instruction 2 

Instruction 2 (ADD 2 x,y) is executed on line 9. because of the IF-statement on line 8. It places a sequence of 
25 characters, which is of length rt s" (which equals 2) at the location VERSION2 + c. The pointer is updated on line 16. 

Instruction 3 

Instruction 3 (COPY 6 20) is executed on line 14, because of line 12: "p" (obtained from the instruction, and which 
30 equals 20) is less than "n." The copy function uses VERSION2 as the source, beginning at VERSION2 + c. The copy 
function takes a sequence which is s characters long. (The variable "s" is obtained from pointer-minus-20. The "20" is 
obtained from the instruction.) 

Instruction 4 

35 

Instruction 4 is executed on line 13, like instruction 1 . 

Generation of TRANSFORMATION 

40 Figure 5 contains a procedure which examines each VERSION, character-by-character, and generates the set of 
instructions which allows construction of VERSION 2 from the combination of VERSION 1 and the instructions, which 
are called a transformation. The transformation of Figure 2 was discussed above. The general approach taken by this 
procedure is the following. 

45 Run 1 

VERSION 1 is processed first. VERSION 1 is shown at several locations in Figure 2F. The code of Figure 5 asks 
whether the four-character sequence beginning at position 0 matches a four character sequence beginning at a previous 
position. Of course, since nothing exists prior to position 0, the answer is No. Thus, position 0 is flagged with a carat, as 
so indicated in the row labeled RUN 1. 

Run 2 

On the second run, the code asks a similar question, namely, whether the four-character sequence beginning at 
55 position 1 matches a four-character sequence beginning at a previous position. The answer is No. and a flag is placed 
at position 1. 
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Runs 3 and 4 

In a similar way, flags are placed at positions 2 and 3, resulting in the flagging shown at RUN 4 in Figure 2F. 
s Run 5 

Run 5 produces different results. In RUN 5. the code asks the usual question, Does the four-character sequence 
beginning at position 4 match a four-character sequence beginning at a previous position ? The answer is Yes, at position 
0, as indicated in the row labeled RUN 5. Position 4 is not flagged, and the EXTEND function (line 16, Figure 5) is called 
10 into action. 

The EXTEND function asks how long the match extends. It asks whether the next position after the matched block 
(the matched block occupies positions 4 - 7, so the next is position 8) has a match located four positions earlier. The 
answer is Yes, as indicated in trial TA. 

It then asks whether the second position after the matched block, position 9, has a match located four positions 
75 earlier. The answer is again Yes, as indicated by trial TB. This inquiry continues until a position is reached, in trial TE, 
where no such match is found. 

The EXTEND function has thus ascertained that the match extends for eight positions, from position 4 through 
position 1 1 . Because of the logic of the code, the last three positions of the extended match are flagged, namely, the 
"BCD" located in "RUN 5 RESULT." 

20 

Run 6 

Run 6 asks whether the four-character sequence beginning at position 12 (ie, the rightmost "e") matches any four- 
character sequence beginning at a position occurring previously. The answer is No, and a flag is placed at position 12, 
25 as indicated in the row labeled RUN 6. 

Results 

The results are shown at the bottom of Figure 2F, and labeled RESULT. Two important features are the following: 
30 One, as will be later explained, the flags indicate not only the beginnings, but also the endings, of possible' 

sequences which are used to construct VERSION 2. 

Two, there are regions which are not flagged, as indicated. A search for sequence to use in constructing VERSION 
2 never begins in these regions, although the search can begin outside these regions, and then invade the region. This, 
since initial search points are eliminated, the overall searching time is reduced. 

35 

Processing of VERSION 2 
First Four Runs 

40 VERSION 2, the updated version, is processed next. The code asks whether the four-character sequences beginning 
at positions 16,17,18, and 1 9, shown in Figure 2G, have matches which begin at previous positions. The four-character 
sequence beginning at position 16 matches with that beginning at 0, but the latter three do not, so positions 1 7, 18, and 
19 are flagged, as indicated at the top of Figure 2G. 

The EXTEND function finds no extended match. The code of Figure 5 jumps to line 29, where it issues the "COPY 
45 4 0" instruction, which was explained above, and appears in Figure 2. 

The overall results are indicated by the arrow labeled OUTPUT. Three characters have been flagged, and a COPY 
instruction has been issued. It is important to note that these flagged characters are later treated as search initialization 
points, just as are the flagged characters in VERSION 1 . 

50 Run 5 

The code of Figure 5 asks whether the four-character sequence "xyxy", beginning at position 20 (which resides in 
VERSION 2, shown in Figure 2G, in the section labeled RUN 5), has a match which begins at a previous position. The 
code searches, starting at each flag, including those now existing in VERSION 2. 
5$ Further, at each flag, the code searches both forward and backward. (This forward-backward searching was not 
mentioned previously, for simplicity.) The overall searching is indicated by RUN 5 in Figure 2G. Because the searching 
(a) starts with each flag and (b) searches for four characters, both forward and backward, every four-character sequence 
of trial T1 through T1 2 is examined. In addition, similar searching is done, starting with the flagged positions in VE RSION 
2, namely, positions 17, 18, 19, and 20. 



3DOClO.<EP 0717353A2J > 



EP0717 353 A2 

No match is found, so the Y at position 20 is flagged. 
Run 6 

5 The code of Figure 5 asks whether the four-character sequence "yxyx", beginning at position 21 (which resides in 

VERSION 2), has a match which begins at a previous position. As for position 20, the code searches, forward and 
backward, beginning at each flag, including those in VERSION 2. No match is found, so position 21 is flagged. The 
current flagging is indicated at the upper right part of Figure 2H. 

10 Run 7 

The code of Figure 5 asks whether the four-character sequence "xyxy", beginning at position 22 (which resides in 
VERSION 2), has a match which begins at a previous position. The answer is Yes, at position 20. Thus, position 22 is 
not flagged, as indicated in Figure 2H, and the EXTEND function (line 16 of Figure 5) is entered. 
15 The EXTEND function determines, in the manner described in connection with Figure 2F, that the match extends 
through position 27 in Figure 2H. Consequently, the last three positions of the extended match are flagged, resulting in 
the flagging shown in the box at the lower left of the Figure. 

At this time, the logic proceeds to line 29 of Figure 5, wherein the two instructions "ADD 2 x,y" and "COPY 6 20" 
are issued. (The arguments "add" and "c" in line 29 refer to the ADD instruction. The arguments "pos" and "len" refer to 
20 the COPY instruction.) 

Run 8 

The code asks whether the sequence "bcde", beginning at position 28 (See OUTPUT box in Figure 21), has a match 
25 beginning at a previous position. The answer is Yes, at position 9 in VERSION 1 (see Figure 2G, upper left). Now the 
EXTEND function is called, which determines that the match extends to an additional character, the T. 

The code, at line 29, then issues the instruction "COPY 5 9". The four instructions of Figure 2 have been generated. 
VERSION 2 can now be reconstructed from VERSION 1. plus the instructions. 

30 Characterization 

The preceding overview illustrates the following principles: 
ONE, once it is determined that a position represents a previously matched sequence, that position is not flagged, 
and consequently, subsequent searches are not initiated at that position, because such initiation would be redundant. 
35 Thus, a reduced set of searching positions is obtained (if possible). 

TWO, VERSION 2 is constructed from two types of sources, namely, (a) sequences copied from either VERSION 
1 or VERSION 2 and (b) sequences ADDED to VERSION 2. Further, the ADDED sequences can be used later, in 
copying operations. 

From another point of view, if a first user possesses VERSION 1 , and if a second user possesses both VERSION 
40 1 and VERSION 2, the first user can construct a replica of VERSION 2, if the second user identifies the following 
sequences of characters: 

(a) those to be copied from the first user's VERSION 1 , (b) those to be added to first user's VERSION 1 , and 
(c) those to be copied from the first user's replica of VERSION 2. 

45 

THREE, each flagged position represents the starting point of a possible vocabulary word. Analogous words are 
shown in Figure 1 A. However, unlike the vocabulary of Figure 1 A, the vocabulary of each flagged position represents 
numerous possible sequences. 

For example, assume VERSION 1 is the following: 
so abcdeFghijk. 

If the upper-case F (position 6) is flagged, then it can represent the following sequences: 

f fg fgh fghi fghij fghijk. 
These sequences can contain other flagged positions. 

Therefore, a singlef lagged position can represent multiple sequences for copying into VERSION 2. Thus, thisf lagged 
55 position can appear in multiple COPY instructions. For instance, "COPY 3 6" means COPY "fgh." "COPY 5" means 
COPY "fghij". 
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Security Aspects 

As mentioned above, the ADD instructions contain information as to the content of VERSION 2, and this information 
may need security protection. One approach to security is shown in Figure 1J. 
5 The ADD instruction of Figure 2 ("ADD 2 x,y"), which is sent, is modified to contain a pointer to a location within 

VERSION 1 . This modified instruction is labeled INSTRUCTION SENT in Figure 1J. This pointer is "2". which indicates 
the "c". 

The information sent (ie, the bytes representing V and "y") is * hen EX-ORed with the data beginning with the "c". 
Figure 1J, left side, indicates the EX-OR operation. The instruction sent is "ADD 2 2" plus the result of the EX-OR 
10 operation. This EX-OR result contains no information of value to an eavesdropper, because the eavesdropper has no 
access to VERSION 1. 

The data contained in the received instruction, indicated on the right side of Figure 1K, is EX-ORed with the same 
data in VERSION 1 , namely, that which begins with the "c". This EX-ORing recovers the original data, namely "x.y". 
This procedure exploits a property of the EX-OR operation, namely, that EX-ORing a first word with a second word 
75 produces a third word. ED-ORing the third word with the second recovers the first. 
Thus, from the transmitted instruction, which is 
ADD 2 2 [EX-OR result], 
one obtains 

ADD 2 x,y, 
20 which is the intended instruction. 

Important Considerations 

1 . The order in which the COPY and ADD instructions occur is, of course, important. If they are scrambled as to 
25 order, a different VERSION 2 will be obtained. 

From another perspective, the sequence itself is a type of information. The sequence can be viewed as a per- 
mutation of the instructions. The permutation contains information, just as permutations of the English alphabet 
create words, which convey information. 

Of course, the instructions could be transmitted in a scrambled order, if information is included which allows 
30 reconstruction of the sequence. For example, each instruction could be numbered. 

Irrespective of how execution of the proper sequence of instructions is attained, the execution produces a replica 
of VERSION 2, by a concatenation process. That is, returning to the example of Figures 2B through 2E: 

1 . First "abed" is written into the replica (Figure 2B). "abed" is taken from VERSION 1 . 
35 2. Then "xy" is concatenated (Figure 2C). "xy" is obtained from VERSION 1. 

3. Then "xyxyxy" is concatenated (Figure 2D), "xyxyxy" is obtained from VERSION 2. (Alternately, four pairs of 
"xy" could have been taken from VERSION 1 and concatenated in step 2. This would be less efficient, however.) 

4. Then "bodef" is concatenated (Figure 2E). "bedef " is obtained from VERSION 1 . 

40 2. Further regarding Consideration Number 1 , above, the Inventors point out that each COPY and ADD instruction 
contains the length of the passage to be COPIED or ADDED. Thus, for any given COPY or ADD instruction, the 
address within VERSION 2 where the COPYING or ADDING occurs can be readily computed, based on the total 
preceding lengths. (The code of Figure 3 illustrates this, because the pointer is computed based on these lengths.) 

45 3. The invention can be used to update any type of data file. It is not limited to text files, or to files of other specific 
types, ft can handle binary files generally. 

Files contain characters. Each character is commonly represented by a byte of data. Since a byte contains eight 
bits, 29, or 256, possible characters can be represented by one byte. 

In "text" files, all of these possible combinations are not used. Only those representing alphanumeric, and 
so some punctuation, are used. 

In "binary" files, all possible 256 combinations are used. The invention can handle binary files. In contrast, 
differencing programs of the prior art can handle only text files. 

4. The invention is not limited to remote replication of VERSION 2 from a stored VERSION 1 . In addition, a later 
55 version of a program can be stored at a single location, by using the ADD and COPY instructions, rather than storing 
the version in its entirety. This approach will save storage space. When a version is desired to be reconstructed, 
the program of Figure 3 is executed. 



5DOCIQ <EP 0717353A2_I_> 



9 



1 



EP0717 353 A2 

5. It is not necessary that the version of a file which is reconstructed be the actual, chronologically later version of 
an earlier file. For example, VERSION 1 can be reconstructed from VERSION 2. 

Technical Discuss! n 

5 

A more technical discussion of the code given in Figures 3 and 5 will now be given. 
Overview 

10 A data file can be considered as a sequence of bytes. On virtually all current computers, a byte is the smallest 
natural unit that can be compressed efficiently in terms of storage, communication, and in-memory data manipulation. 
Though the term "data file" often refers to a file when stored on-disk, the present invention also allows such a sequence 
of bytes to be some segment of main memory. 

Figure 1 shows an example of using the present invention to keep data in synchronization between two computers. 

is First, on the source computer, the two versions of the data, VERSION 1 and VERSION2, are compared to produce a 
transformation that takes VERSION1 to VERSION2. This transformation is transmitted via some communications chan- 
nel to the target computer. Then, on the target computer, the transformation and the local copy of VERSION1 are used 
to reconstruct VERSION2. 

20 Coding and Decoding Transformation Instructions 

The transformation computed by the present invention consists of a sequence of two types of instructions, COPY 
and ADD. During the reconstruction of VERSION2, the COPY instruction defines the position and length of some existing 
segment of data to be copied while the ADD instruction defines a segment of data to be added. Figure 2 shows an 

25 example of two data files in which VERSION 1 consists of the sequence of bytes "abc dab cda bed efgh" (spaces are 
added here for readability). VERSION2 consists of the sequence "abc dxy xyx yxy bedef Thus, VERSION1 has length 
16. and VERSION2 has length 17. The transform shown in Figure 2 shows the instructions needed to reconstruct 
VERSION2 from VERSION1 . Note that we use the convention that positions in VERSION1 are coded starting from 0. 
while positions in VERSION2 are coded starting from the length of VERSION1 . For example, the third instruction of the 

30 transform in Figure 2 is a COPY instruction that copies 6 bytes starting from position 4 of VERSION2, which is coded 
as 20. 

Figure 3 shows the procedure of how the reconstruction of a version is carried out in general. Line 1 initializes the 
variable "n" to the length of VERSION 1 . Une 2 sets the current position "c" in VERSION2 to zero. Line 3 starts a loop 
that is terminated on line 6 after the end-of-f ile condition is detected in line 5. Line 4 calls the function readinstO to read 
35 an instruction. Line 7 reads the copying size or the data size using the function readsizeO- Lines 8 and 9 check to see 
if the instruction is an ADD instruction and, if so, reads the data into VERSION2 starting at the current position c. Lines 
10 to 15 process a COPY instruction by reading in the position code, then performing the copy operation either from 
VERSION1 or VERSION2, as appropriate. Line 16 increases the current copy position in VERSION2 by the length of 
the newly reconstructed data. The copyO function is a simple function to copy data from one area of disk memory (or 
40 main memory) to another. However, the readinstO, readsizeO, readposO, and readdataO functions must be defined, 
based on some concrete definitions of how the COPY and ADD instructions and their parameters are encoded. 

Applying the procedure of Figure 3 to the example of Figure 1 , we see that there are four steps in the decoding. 
The first step copies four bytes "abed" from VERSION1 , starting at position 0. THe second step adds 2 data bytes "xy". 
The third step copies from VERSION2 6 bytes starting at its position 4 (counting from 0 and coded as 20 by our con- 
45 vention). Note that at the start of this step, only the first 6 bytes "abedxy" of VERSION2 have been reconstructed as only 
the two bytes "xy" are available to copy. However, since data is copied from left to right, whenever a byte is to be copied 
it will have been constructed. The fourth and final step copies the 5 bytes "bedef " from position 9 of VERSION1 . 

We now give a description of the encoding of COPY and ADD instructions. This particular embodiment of these 
instructions are chosen since in the Inventors' experimentation they perform well for many different types of data. Given 
so the description, the readinstO, readsize(), and readpos() functions above can be straightforwardly implemented. Each 
instruction is coded starting with a control byte. The eight bits of a control byte are divided into two parts. The first four 
bits represent numbers from 0 to 1 5, each of which defines a type of instruction and a coding of some auxiliary information. 
Below is an enumeration of the first 10 values of the first four bits: 
0: an ADD instruction 
55 1 .2,3: a COPY instruction with position in the QUICK cache; 

4: a COPY instruction with position coded as SELF; 
5: a COPY instruction with position coded as difference from HERE; 
6,7,8,9: a COPY instruction with position coded from a RECENT cache. 
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The QUICK cache is an array of size 768 (3 x 256). Each index of this array contains the value p of the position of 
a recent COPY instruction such that p modulo 768 is the array index. This cache is updated after each COPY instruction 
is output (during coding) or processed (during decoding). A COPY instruction of type t, 2, or 3 will be immediately 
followed by a byte whose value is from 0 to 255 that must be added to 0, 256, or 512, respectively, to compute the array 
5 index where the actual position is stored. 

A COPY instruction of type 4 has the copying position coded as a sequence of bytes. 

A COPY instruction of type 5 has the difference between the copying position and the current position coded as a 
sequence of bytes. 

The RECENT cache is an array having 4 indices, and stores the most recent 4 copying positions. Whenever a COPY 
io instruction is output (during coding) or processed (during decoding), its copying position replaces the oldest position in 
the cache. A COPY instruction of type 6 (respectively 7, 8, 9) corresponds to cache index 1 (respectively 2] 3, and 4). 
Its copying position is guaranteed to be larger than the position stored in the corresponding cache index and only the 
difference is coded. 

For the ADD instruction and COPY instruction of type from 1 through 9, the second 4 bits of the control byte, if not 
75 zero, code the size of the data involved. If these bits are 0, the respective size is coded as a subsequent sequence of bytes. 
It is a result of the encoding method that an ADD instruction is never followed by another ADD instruction. Frequently, 
an ADD instruction has data size less than or equal to 4, and the following COPY instruction is also small. In such cases, 
it is advantageous to merge the two instructions into a single control byte. The values from 10 to 15 of the first 4 bits 
code such merged pairs of instructions. In such a case, the first 2 bits of the second 4 bits in the control byte code the 
20 size of the ADD instruction and the remaining 2 bits code the size of the COPY instruction. Below is an enumeration of 
the values from 10 to 15 of the first 4 bits: 

10: a merged ADD/COPY instruction with copy position coded as SELF, 
1 1 : a merged ADD/COPY instruction with copy position coded as difference from HERE; 
12J3.14.15: a merge ADD/COPY instruction with copy position coded from a RECENT cache. 
25 Figure 4 shows an encoding of the four instructions of the transform shown in Figure 2. For each instruction, all 
eight bits of the control byte are shown. For example, the first four bits of the second control byte are zero, showing that 
the byte codes an ADD instruction. The second four bits of the same byte show that the size of the ADD instruction is 
2. The two data bytes "xy" come after the control byte. All three COPY instructions are coded using the SELF type, 
meaning that their copying positions are coded in bytes (with shown values) following the control bytes. The size param- 
30 eters of all the instructions are small enough so that they can all be encoded inside the control bytes. This small example 
shows that VERSION2 which is 17 bytes long can be coded in a transformation using only 9 bytes. 

Fast Computation of Matched Segments 

35 Figure 5 shows the method to partition VERSION2 into segments and encode them as COPY and ADD instructions. 
Note that, for this encoding method, matches of short lengths are not useful because they take up at least as much 
space to encode as the data that they match. So the matching method can be tuned to ignore short matches. The 
minimum match length that we use is 4 and represented by the variable MIN in the body of the encoding procedure. 
Line 1 of Figure 5 initializes a search table T to empty. For efficiency, T is kept as a hash table with chaining for 

40 collisions. This data structure is standard, and is described in any data structure and algorithm textbook, such as pages 
111 - 112 in "The Design and Analysis of Computer Algorithms" by A. Aho, J. Hopcroft, and J. Ullman, published by 
Addison-Wesley, 1974. 

Table T contains certain selected positions in the two versions. These positions are inserted with the procedure 
insertO and used in the procedures searchO and extend() for fast searching of longest matched data segments. Lines 

45 2 and 3 call the procedure processO to select positions from VERSION1 and VERSION2 to insert into the table T COPY 
and ADD instructions are also generated during the processing of VERSION2. 

Lines 4 to 45 define the procedure process(). Lines 5 and 6 initialize the variables "n" and "m" to the length of 
VERSION1 and the version being processed. Line 7 initializes the curent position "c" of the version being processed to 
0. Line 8 initializes the start of data for an ADD instruction to -1 (ie, none). Lines 9 and 10 initialize the position and 

so length of the longest match for the data segment starting at position c Lines 1 1 to 42 define the main loop that process 
the given version. Lines 12 to 18 compute a longest, already-processed data segment that matches with the data seg- 
ment starting at c. The procedure searchO called on line 12 finds a match of length len+1. This procedure uses MIN 
bytes starting at position c+Jen-(MIN-1 ) as a key to search for a matching position in the table T Then it matches backward 
the data in *seq" and the data in the appropriate version to see if the match covers everything from c to c+len+1 . 

55 If such a match is found, the procedure extend() called on line 16 extends the match as far forward as possible. 
Lines 14 and 15 correspond to the case when there is no match and we break out of the search loop. Otherwise, the 
loop is repeated to find a longer match. Lines 19 and 26 insert the current unmatched position c into the table T 
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Note that the actual value to be inserted is c if seq is VERSION1 ; otherwise, it is c plus the length of VERSION1 
(which is the convention for encoding positions in VERSION2thatwe described earlier). The procedure insert(T,seq,p,ori- 
gin) inserts the coded position p+origin into table T using as its key. MIN bytes starting from position p in version seq. 
Lines 20 and 21 set the variable add to c, if it is not yet defined, to indicate that this is the beginning of a segment 
5 of unmatched data. Line 25 moves the current position forward by 1 for the next iteration of the processing loop. Lines 
27 and 28 correspond to the event of finding a longest matched segment. Lines 28 and 29 call the procedure writeinstO 
to write out the COPY and ADD instructions according to the description given earlier. 

The first two arguments to writeinstO define the data for an ADD instruction if add is zero or positive. The second 
two arguments define parameters for the COPY instruction. The working of this procedure is straightforward so we shall 
w omit its description. Lines 30 to 36 insert into table T MIN-1 positions at the end of the matched data segment. Lines 37 
and 38 increase c by the length of the match, and resets add to -1 . Lines 40 and 41 terminate processing if there is not 
sufficient data to be processed. Lines 43 and 44 output any final unmatched data from VERSION2 as an ADD instruction. 

Figure 6 shows a procedure to insert a position into the table T. Line 2 of Figure 6 indicates that the key is the MIN 
bytes of the sequence "seq" starting at the position p. Line 3 constructs the coded position. Line 4 inserts the pair (key, 
is pos) into the table T 

Figure 7 shows a procedure to search for a match. Lines 3 and 4 construct the search key, which consists of MIN- 
1 bytes at the end of the current longest match length, plus a new unmatched byte. Lines 5 to 19 try to extend the match 
length if possible. This is done by looking at every element of the table T that matches the constructed search key and 
seeing if the part of the current match left to the key position matches with the corresponding part of the element being 
20 considered. This test is carried out on line 1 7. If this is true, then searchO returns the starting position of the match on 
line 18. Line 20 returns "-1" to indicate that there is no match longer than "len M . 

After the call to searchO is successful on line 13 of Figure 5, the match length is now known to be at least 1 more 
than th "len" value in Figure 5. Figure 8 shows the extendO procedure, which extends the match to the right as far as 
possible. Lines 2 to 10 set up the correct sequences to search for matches, lines 11 to 13 perform the extension, line 
25 14 returns the entire matched length. 

When applied to the example of Figure 2, the above method will compute exactly the set of instructions exhibited 
in the same figure. Figure 6 shows the positions in the example VERSION1 and VERSION2 that would be inserted into 
table T. 

30 Security Enhancement 

Given two versions, VERSION1 and VERSION2, and a computed transformation, only the ADD instructions in the 
transformation carry raw data from VERSION2. Such data can be utilized by a snooper to gain valuable information 
about VERSION2. In applications having high level of security requirements, to prevent leakage of information, the ADD 

35 instruction can be modified as follows. First, each ADD instruction is modified to also have a copying address which 
would be some position randomly selected from VERSIONI in such a way that the data segment starting at that position 
is at least as long as the length of the ADD data. If this is not possible, the ADD data can be broken into smaller units. 
Then, the raw data is xor-ed with data from such a selected segment of data before output to the transformation. For 
example, using the same ADD instruction in Figure 2, if the selected position in VERSION1 was 2, then this position 

40 would be output immediately after the control byte, and the two data bytes "xy" would be xor-ed with the two bytes "cd" 
before output. 

On decoding, the same xor operation must be done to each data byte before outputting. This works because the 
mathematical property of xor guarantees that when a byte is xor-ed twice with another same byte, it retains its original 
value. However, we can now guarantee that a snooper will not be able to gain any information from the transformed data 
45 bytes unless he or she already has a copy of VERSION1 , which was assumed to be secure. 

Figure 7 shows the instruction of Figure 4 encoded using this more secure method. The ADD instruction now has 
a position 2. The data bytes "x" and V are shown being xor-ed with "c" and "d" respectively. 



50 



55 



12 

iDOCID: <EP 0717353A2_I_> 



EP 0 717 353 A2 



Tue Aug 30 09:21:37 1994 



2: 
3: 
4: 
5: 
6: 
7 : 
8: 
9: 
10: 
11: 
12: 
13 i 
14: 
15: 
16 i 
17: 
18: 

19 : 

20 : 
21: 
22: 
23: 
24: 
25: 
26: 
27, 
28: 
29: 
30: 
31: 
32: 
33: 

34: 
35: 
36 : 
37: 
38: 
39: 
40: 
41: 
42: 
43: 
44 : 
4S: 
46: 
47: 

48 : 

49 : 

50: 
51: 
52: 
53: 
54. 
55: 
56: 
57: 
58: 
59: 



APPENDIX 
(24 pages) 



I oeraor :ng . zoo . , com i /n/arvDhQr./a? Apv/snf r>. / ^r^/i hyv^> - B ■ 

•lfndfcf _VDELTA_H 
# define _VDEXTX_H 1 

•ifndef KPV 

•define _KPV i 

•ifadef STD_C 

•ifdef 9TOC 

•define STD.C 1 

• else 

•if cplusplua 

•define 3TD.C 1 

#•130 

ttdefir^ STD_C 0 

•endif /* fc cpluepiu««/ 
•endif /• STDC */ 

• endif /• STD_C«/ 

•ifndef _BEGI»_EXTER>fS_ 
»lf cpluaplua 

■define _BEGIK_EXTERNS_ extern "C { 
•define _EjrD_EXTER-V9_ j 
•elae 

•define _JBEGIN_EXTERNS_ 
•define _EKD_OCTERS6_ 
•endif 

• end If /• .BEGIN. DC TERNS.*/ 



•ifadef _AAC_ 

• if STO_C 

•defiae _ARC_(X) 
•elae 

•define ^AEC.(X) 
•endif 

•endl: /*_ARG_V 



( ) 



•ifndaf void.t 

• if STD_C 

• define VoitfL.t 

• else 

•defino Void.t 
•endif 

ttendir /•voider*/ 

•ifndef Nil 
•define NIL (type) 
•endif /*n:lv 

• endif /* KPV */ 



void 
char 



((typ«)0) 



/* uaer-eupplied function* to do io */ 
typedef struct .vddiio,a Vddiso_tr 

typedef int(* vdio^f J^ARG.t ( int, Void_tV int, long, vddiac.fjjj 
atruct _vddiec__ a 

( vdio_f readf: /• to reed data 

Vdio_f writef; /• to writ* data 

Ion? window; /* window size if any 

); 



V 
V 
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t saragr -nft aoa at*l com l /n/arynhQn/a7 /Vcv/so f tw&re /ars/lib /vsa 1 ta /vdel: a h 



60: /» typee that can ba given to the 10 functions */ 

61*. tdefine VD_S0URCE 1 /* io on the source data */ 

63: f define VD — TARGET 2 /• io on the target dara */ 

63: udefine VD^PELTA 3 /* io on the delta data */ 



64: 

10 65: /* magic header for delta output */ 

66: 'define VD^, MAGIC "vdOl* 



67 : 

68 : _BEGIN_EXTERK3_ 

69: extirn long vddelta _ARG_( <Void_t* , long, void^t* , long/ Vddise_t*>); 

70: extern long vdupdate _>RG_( (Void.-* , long, Void_-*, long, vddiac_r* ) > : 

7 1 : .EKD.EXTSRNS. 
15 72: 

73.- #endif /*_VDELTA_HV 
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SO 



2: 
3: 
4 : 
5. 
6: 
7 : 
6: 
9 : 
10: 
11: 
12: 
13: 
14 : 
15: 
16: 
17 j 
IB: 
19: 
20: 
21: 
22: 
23: 
24 : 
25: 
26: 

27 i 

28 i 
29i 
30: 
31i 
32: 
33: 
34: 
35: 
36: 
37: 
36 : 
39: 
40: 
41; 
42: 
43: 
44: 
45: 
46: 
47: 
48: 
) 

49 t 
50: 

) 

SI. 
52: 

53 : 

54 : 

55: 
56: 
57 : 



*ifndef _VDELHDR_H 

< define .VDEIHDR^H 1 

< include "vdelta.h" 

♦if STD_C 

* include <stddef.h> 
• else 

•include <aya/typee . h> 
•endif 

•ifdef DEBUG 
_BBCIN_EXTERNS_' 

extern int abort ( ) ; 

_END_EXTERNS_ 

♦define ASSERT(p) ((p) ? 0 : abort<)) 
•define DBTOTAL( t, v> ((t) (V) ) 

•define DBMAX ( m , v ) (<ra) - <ra) > (v> ? (m) : (v> ) 

telae 

♦define assert(p) 

•define DBTOTAL{ t , v) 
#define DBMAX (m,v) 
•endif 

/* short -hand notation* •/ 
•define reg regiater 
•define ucfcar unsigned char 

•define uint unsigned int 

•define ulong unsigned long 

/* default window size - Chosen to suit mslloc() even on 16 -bit machines, 
•define MAXINT ( ( int ) ( < (uint) -0) » 1)) 

•define MAXWINDOW ( (int) (( (uint) -0) » 2)) 

•dsfins DFLTMINDOW (KAXWINDOW <• <1«14) 7 ( 1«14 ) : (1«16) 

•define HEADER (w) ((w)/4) 



•define M_MIM 



/* rain number of bytea to match */ 



/* The hash funotion is sC0]*alpha~3 + • (1) *alph* A :i + «[9l*Alpha «■ a HI V 
•define * T.PKA 33 

• if 0 

•define Al(x,t> (ALPHA* (x)) 

• define A2(X,t) ( ALPHA* ALPHA* ( x > ) 
•define A3(X,t) (ALPHA*AX.PH**ALPHA*(X) > 

• elae /* fast multiplication using afciftatadda •/ 



•define Al(x,t) 

•define A3(x,t) 

•define A3(x,t) 

•endif 

•define HIKIT(h,a,t) 

•define HNSXT(h,e.t) 

•define EQUAL t a. t) 



((t - (x) ) , <t * ct«5) j ) 
(<t - (X)), (t ♦ (t«6) + (t«10)) > 
((t - (X)), (t r (t«5) r ( (t+(t«4) )«6) 



( ( tW ' <<-i • J <<1 



((h - A3(a[0J,t>), (h t- A2(a(l],t>), (h — Al ( a [ 2 ] . z ; -b ( 3 
<(h A3<e(-1] ,t) ), (h - Al<h,t> ♦ a 1 3] ) ) 



((•)[0] « (t)(0] &4 (a)EU (t)(15 44 \ 
(S)C2J — <t)[2] 4& (s>{3] — (t)C31 ) 

/• Every instruction will atart with a control byte. 
For portability, only 8 bits of the byte are uaad. 
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74 : 
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76 : 
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37 : 
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96 : 

97 : 

98 : 
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100: 
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102 : 

10 3: 
104: 
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110: 
111: 
112: 
113: 
114: 
115: 
116 

in 



The bita are used as follow*: 

:::: ssss 
sasi: size of data involved. 
1 1 1 1 .- this defines 16 instruction types: 

0: an ADD instruction. 

1,2,3: COPY with K.QOICK addressing schema. 

4,5: COPY with K — SELF * K__HERE addressing schemes . 

6,7.8,9: COrU rfith K_RECENT addressing scheme . 

Tor the above types, 8988 if not zero codes the size: 
otherwise, the size is coded in subsequent bytes. 

10,11: merged ADD/COPY with K.SELF. K.HERE addressing 

12-13,14.15: merged ADO/COPY with !£_ RECENT addressing - 

For merged ADD/COPY instructions, asss is divided .r.tc "cc aa" 
where cc codea the size of COPY and aa codes the s*_ze of ADD. 



•define VDJBITS 

•define S_BITS 
•define t_BIT9 



* bits usable in a byes 

bits for the size field 

birs for the instruction type 



/ 



The below macros compute the coding for a COPY address 
** There are two caches, a "quick" cache of ( K_QTYFE* 2 56 ) addreaaes 
** and a revolving cache of K__RTYPE "recent" addresses. 
** First, we loo* in the quick cache to see if the addrees ia there. 
*» If so, we use the cache index as the code. 
** Otherwise, we compute from 0, the current location and 

** the "recent'' cache an address that is closest to the being coded address, 
*• then code the difference. The type is set accordingly. 



An invariancej is 2*K_M£RGE + K_QTYPE 
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V 

•define X.RTYPE 

•define X_QTYPE 

•define MERGE 

•define K_QSI2S 

• define K_Q'JICK 
•define K.SELP 
•define K_H£RE 

• define K_RECENT 



4 / 

3 / 
(K_RTYPE+2) / -- 

(K_QTYPE«VD.3ITS) /* size of 



# Of K — RECENT types •/ 

# Of K.QU1CK types */ 

# of typea allowing add-copy */ 
* K_aOTCK cache »/ 



1 /* start of K_QUICX cypas 

( K.QUICK+K.OTYPE) 

(K.SELF+l) 

(K_KERE+1) /* etart Of RECENT types */ 



•define KJJDECL (quick, recent, rhere) /• cache decls in vdelta 

int quicx(K_QSIZEl ; int r«>cent IK_RTYPE] ; int rhere/*;*/ 
•define K_t^ECL (quick, recent , rhere) /* cache decls in vdupdate 

long quick ( K_Q3tiB] ; long recent ( K_RTYP£ ] ; int rhere/*;*/ 
•define K.INIT( quick , recent , rhere ) \ 
( quick (rhere-0 J - <1«7); \ 
while* (rhere +- 1) < K.QSIZE) quick:rhere] - rhere * 1 1«7 > ; \ 
re ce nt ( rhe re-0 ] • ( l«d ) ; \ 

while ((rhere 1) < K_RTxF£) recent ( rhere] - ( rhereri) • ( 1«8 ) ; 

) 

• define K_UPD ATE (quick, recent, rhere , copy ) \ 
f quick (copy%K_Q3IZE] - copy; \ 

if (trhere t- 1) >- K.RTYPE) rhere - 0; recent [ rhere ] - copy; \ 



*/ \ 
*/ \ 



\ 



} 



•define VD_I3CCPY(k) 
•define K_ISMERCE(k) 



( (It) 
( (k> 



> 0 a !)C) < (X_RECENT-K_RTYPE) ) 
>- (K_RECSNT+K_aTYPE) i 
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118: 
119: 
120: 
121: 
122: 
123: 
124: 
125: 
126: 
127: 
128: 
129: 
130: 
131: 
132: 

133: 
134: 
135: 
136 ; 
137: 
138: 
139: 
140: 
141: 
142: 
143: 
144: 
145: 
146: 
147: 

148 : 

149 : 
150: 
151: 
152: 
153: 
154 : 
1S5: 
156 : 
157: 

158: 

159: 

160: 
161: 
162: 
163 : 
164: 
165: 
166 : 
167: 

168 : 

169 : 
170: 
171; 
172: 
17 3 t 
174 : 
175: 

176: 
177: 



•def in* 


A_3I2E 




•define 


\_1310CAL(8) 


•define 


A_1PUT( 9 ) 




•define 


A__?UT<fl) 


( (fl) 


•def ine 


A_ISHERE(1) 


#def lne 


A_LCET( 1 ) 


<(1> 


•define 


A_GET{8) 


( (a) 


•define 


C_SIZE 




•define 


C_rSLOCAL(«) 


•define 




((■) 


•define 


C_PUT(a) 


M«) 


•define 


C_I3HERS(i) 


•define 


C_tGET(i) 


<<U 


•define 


C_GET(a> 


( (8) 


•define 


K_?UT(k) 


( (k) 


•define 


K_CET { i ) 


( (i) 



< <1«S_BIT3) -1) 
( (S) <- A_SII£ 



((1) & ASSIZE) 



/• isax local ADD sue 
/* can be ceded Iccali 

/• ceded local value 

/• coded noraal value 

/• locally coded s::e 



{ (1«S_BITS) *M^MIN-2) /* TAX local COPY size */ 



( (») <- C_SIZE > 
(M_MIK-1) ) 



/* can be coded lecally •/ 



/• ceded local value 



((i) & ( (1«3_BITS)-1) ) /• size was codad local 



/• coding merged ADD/COPY instruction* »/ 

•define A_riNY 2 /* bits for tiny ADD •/ 

•define a.tinysize <1«a_TINY) /• max tiny ADD size •/ 

•define A_ISTINY(s> <<*) <- A.TINYSIZE ) 

•define A_TPOT(i) ((e) - 1) 

•define A.TGET(x) (((i) & ( A_TINY3IZE- 1) ) + 1) 

•define C_TINY 2 /* bita for tiny COPY 

•define C^TINYSIZE ( ( 1<<C_ TINY) - H-MIN-l) /* max tiny CCPY size 

•define C_rSTi;fY(») ( (s) <- C.TIKYSIZE) 

•define C_TPUT(s) (((■) - M__MIM ) << A_TINY) 

•define C_TGET( i ) ((((i) » A_TINY) & J (1«C_TIHY)-1)> - K_>MtN ; 

•define K_IPUT(k) ( < < Jc ) +K_MERCB ) « 3_BIX3) 

•define MEMCPY ( to , from/ n ) \ 
svitch(n) \ 

{ default: memcpy ( (Void_t* ) to , ( Void_t* ) from, ( size. z) n) ; \ 
to -*-- n; from n; break; \ 
case 7 : *to*+ • *from++; \ 
case 6 : •to++ - •from**; \ 
case 5 : *to<*-«- - •from*"*-; \ 
case 4 : *to++ - •from**; \ 
case 3 : *to** • •from**; \ 
case 2 : ♦tor-t* - •from-; \ 
case 1 : •to++ - •from**; \ 
case 0 : break; \ 

) 

/* Below here ia code for a buffered I/O subsystem to speed up I/C •/ 
•define I_9HIFT 7 

• define I.MORE (1«I_3HIFT) /* cor.t ir.ua: ic r. b-< 

•define I_CODE(n) ( ( uchar ) ( ( n) & ( r_MORE- 1 ) ) ) /* get lower bi^s 

/* structure to do buffered IO */ 
typedsf struct _vdio_s 
{ uchar* next; 

uchax # endb; 

Vddiac.t* disc; 
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178 
179 
190 

lai 

182 

183 

184 

185 

186 

187 

188 

189 

190 

191 

192 

193 

194 

195 

196 

19 7 

193 

199 

200 

201 

202 

203 

204 

205: 

206 

207 

208 

209 

210 i 

211 

212 

213 

214 

215 

216 

217 

21fl 

219 

220 

221 

222 



sereori-e zoQ.act.eomi /n/crrvphon/g7 /Vpv /g Q f twarc/arc/llfc /vdel-e/vdelhdr 



long 
uchar 
} Vdio.t; 



here; 
but C1024] 



•define 
•define 
•define 
'define 
•define 
•define 
•define 
•define 
•define 



READF(io) ((io) 

VRITEF(io) 

3UF<io) 

aursizE(io) 
NEXT(io) ((io) 

DISClio) 
HERE(lO) 
RINITUo, 



< (io) 
( (io) 
t (10) 
diac) 
(io> 

•define WINIT< io,diae> 
(io) 

•define REMAIN { io ) 
•define VDPUTC ( io . e > 

•define VDGETC(io) 



■>diae->readf ) 

( (ioj ->diae->writ«f ) 

< {to) ->buf ) 

aizeof ( < io) ->buf ) 
>next ) 
>endb> 
>di«e) 
>here) 

((io)->endb • (<io)->next » (io)->buf). \ 
>here - 0/ (io)->disc • (disc) ) 

((io)->endb - (<io)->next - <io)->buf; BU?SIZE< io ) 
>here - 0, <io)->disc - (disc) ) 

(ENDB(iO) - NEXT(io)) 

((REMAIS(io) > 0 t| <*_Vdf labuf ) (lo) > 0) ? \ 
(int) (•(io)->nextT- - (c)) : -1 ) 
( (REMAIN { io ) > 0 || < *_Vdf ilbuf ) <io> > 0) 7 \ 
(inr) ( *(io) ->r.ext*') : -1 ) 



typedef struct _vdbufio_a 



vdfilbuf )_ARG_< (Vdio.f) ) ; 

vdf labuf ) _ARG_ t { Vdio_t« ) ) ; 
vdge tu ) _ARG_ ( ( Vdio_t * , ulong ) ) 
vdputu)_ARS_( <Vdio_t* , 
vdread)^ARG_< < Vdio_t* , 
vdvrite)_ARG_( (Vdio.t* 
) Vdbufio_t; 

•define _vdf ilbuf _Vdbuf io . vdf ilbuf 
•define _vdf labuf _Vdbufio . vdf labuf 



int( * 
int(« 
ulong ( • 
int<« 
int< < 
int< * 



ulong) ) . 
uchar* - 
ucher* 



int ) ) ; 
int)); 



* define _vdgetu 
•define _Vdputu 
•define _vdread 
•define _vdvrite 

— BEGIN^EXTERNS^. 
extern Vdbufio. t 
extern void_t* 
extern Void^t* 
extern void 
_END_EXTERNS_ 



_Vdbuf to . vdgetu 
_Vdbuf io . vdpu- j 
_Vdbuf io . vdread 
_Vdbuf lo . vdwxite 



_vdbuf io; 

memcpy _arg_( <void_t* , 
melloc _ARG_( <size_t) j , 
t xee _ARG_< (Void.c* ) i ; 



conat void_t», size_t)ji 



• endif /*_VDELHDK_HV 



40 



45 



50 
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1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 



35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
4S 
47 
48 
49 
50 
51 
52 
S3 
54 
55 
55 
57 
38 
59 



i pgraarir.e . zoo att.gom! /n/yr/phon/Q? /kpv/Sof twa?9/arr /lih/vdel- a/vdds 

•include "vdelhdr . h" 

/* compute a trans formation that takes source data to target data 
• * 

«• Written by ( Kiem- ) Phong Vo, kpvcresaarch.att.com, 5/20/94 



*ifdef DE3UG 

long S.copy, S_add 

Icr.g N_copy, >f_add 

long M_copy , M_add 

long K.jasrge; 

•endli 



/* amount of ir.put covered by COPY and AZ 
/* • of COPY and ADD instructions 
/• max size of a COPY or ADD instruction 
/* » of merged instructions 



"define MERCABLE ( a . e , k ) <(a> > 0 44 A_ISTINY(a) it \ 
(C) > 0 64 C_XSTX>TY(C) 64 \ 
(Jc) >" K.SELF ) 



19 


typed*? struct 


_match_» M*teh_t 








20 


typedef struct . 


_table_s Table. t 








21 


struct _match — 3 










22 


C Match_t* 


next; 


/• 


linked list ptr 


•/ 


23 


): 










24 


struct _table_ s 










25 


{ vdio.t 


io; 


/• 


io structure 


*/ 


26 


uchar* 


arc; 


/* 


source string 


V 


27 


int 


n_src; 








28 


uchar* 


tar; 


/* 


target string 


V 


29 


int 


a_tar; 








30 


K_DD£CL (quick , recent , rh*r* ) ; 


/* 


address caches 


V 


31 


Match_t* 


base ; 


/• 


base of elements 


*/ 


32 


i int 


size ; 


/• 


si2S of hanh table 


V 


33 


Match.t** 


table; 


/• 


hash table 


V 


34 


i ); 











/* encode and output dalta instruction* */ 

• if STD_C 

static vdputinat (Table_t* tab, uchar* beg*, uchar* her*, Match_t* match, 

• else 

static vdputinat (tab, b*ga, here, match, n_copy) 
Table.t* tab; 

uchar* begs; /* ADD data if any */ 

uchar* here; /• current location */ 

Match_t* match; /• best match if any */ 

int n_copy» /* length of match */ 

■endif 



int n_copy : 



reg int xuadd, i_add, i_copy, Jc_type; 
reg int n, c_addr, copy, beet, d; 

n_add - begs 7 h*r*-beg* : 0; 

c_addr - ( here - tab- >tar)-»"tab->a_arc; 
k_type - 0; 



/* add axz* 

/* current address 



if (match) /* process th* COPY instruction »/ 

{ /**/t>BT0TAL(N_COpy, 1) ; 0BT0TAL( S_copy , n_copy ) ; OBMAX (M_cc py , r._cc 



best - copy - match 
k-type - R_SELF; 



tab->ba*e ; 
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y*rftgrir.« zoo, act, cam /n/qrvahon/g7 /kov/Sof tvara /arc/lib/vde l-a/rddalta 



60: 
61: 

62 : 

63 : 

64 : 
65: 
66 : 
6? : 

68 : 

69 : 

70 ; 
71: 

72 : 

73 : 

74 : 

75 : 

76 : 

77 ; 

78 : 
79: 
30 : 
81: 
82: 
83: 
84 : 
85: 
86: 
87 : 
88: 
89: 
90: 
91: 
92: 
93: 
94: 
95: 
96; 

97 : 

98 : 
99: 

100 : 
101: 
103: 
103: 
104: 
10 5: 
106 : 
107; 

108 : 

109 : 

110 : 
111: 

112: 

) 

113 : 
114: 

115: 
116: 
117 : 
118: 



if((d - c_addr • copy) < beat) 
{ beat • d; 

k_typs • K_ HERE; 

} 



i or { n 

{ 



0 ; n < K.RTYPE; -*n) 

lf(<d • copy - tab->recent(n) ) < 3 

continue ; 
beat » d; 

•<-type - K_RECENT>n; 



d >= best) 



If (best >• I__MORE && tab->quick[n - copy%K - .QSIZE] copy) 
{ fOX(d - K_QTYPE-1: d > 0; --d) 

if (n >• (d«VD_BITS) ) 
break; 

beet - n • (d«VD_BITS); /»*/A33EHTibe3t < <1«VD_BITS)>; 
k_rype - K_QUICK+d; 

) 

/**/ASSERT(beflt >• 0); 

/**/ASSERT<(Jc.typ«-!*K - MERCE) < (1«I_3ITS) ); 
/* update addresa caches */ 

K_UPDATE(tab->quick, tab->recent, tab- >rhere , copy ) ; 

/* see if rue r gable to laat ADD Instruction */ 
it < MERGABLE ( n_add » n_copy . k_rype ) > 



{ 
> 

else 

( 



/**/DBTOTAI(N_merge, 1) ; 
i_*dd - K_TPUT ( k_type > . A_TPUT ( n_add > , C.TPUT <n_copy 



i_copy - K_PUT ( k_type ) ; 
If (C_ISLOCAI.<n_copy) ) 

i_copy I - C_LPUT(tucc-py) ; 



if(ti_add > 0) 

{ /* * /DBTOTAZ ( N_add , 1 ) ; DB TOTAL ( S.add , a_add ) ; DBHAX ( M.add , n_add ) ; 

if < ! MERGABt S < n_add , n_copy , k_typ« ) ) 

t.add - A_ISLOCAL(tuadd) 7 A_tPUT( n_add) : 0; 

if (VDPUTC< <Vdio_t»)tab,i_add) < 0 ) 

return -1; 
if ( 1A_I3L0Oa(O_add) && 

(*_Vdputu) ( (Vdio^t* )tab, (ulong) A_PUT(n_add) ) < 0 ) 

return -1> 

if ( (•_vdvrite) ( <Vdio_t*)tab, bega, rv_add) < 0 ) 
ra>turn -1; 

> 

if (n_copy > 0) 

( if ( !MERCABM(lV_add,n_copy,k__typ*> 6& VDPUTC ( ( VdlO_t« ) tab, i_ccpy j *C 

return -1; 

if ( :C_;9LOCAL(n_copy) hk 

(*_Vdputu) < (Vdio_t*)tab, ; ulong } C__PUT < ri_copy ) ) < 0 ) 
riturn - 1 ; 
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119 

120 

121 

122 

123 

124 

125 

126 

127: 

128 

129 

130 

131 

132 

133 

134 

135 

136. 

137. 

13 B 
139 
140 
141 
142 

14 3 
14 4 
14 5 
146 
147 
148 
14 9 
ISO 
151 
152 
153 
154 
155 
156 
137 
138 
159 
160 
161 
162 
16 3 
164 
16 5 
166 
167 
168 

16 9 
170: 
171 

17 2 
17 3 
17 4 
17 5 
17 6 
177 
17 8 



perearlr.ft.2QQ.att.eomi /n/ervphon/a? /Vpv/Sof tv*~+ /«~r •.a/vjw-^ ^ 

if<*_*ype >- K.QUICK && fc_-ype < (K_QCICK<-K_0TYPE) ) 
( if (VDPUTC( <vdio_t* j tab. i uchar) beat; < 0 ) 

return -i; 

> 

else 

( If ( <*_vdputu) ( (Vdio_t* >tab, (ulong)fcest ) < D ; 

return -1; 

} 



) 

else 

( 

) 



if ( <*_vdflabuf ) <<Vdic_t*)tab) < 0) 
return -1; 



return 0 ; 



/* Fold a string •/ 
#±f STD_C 

static vdfold(Tabla_t* tab, int output) 
•else 

static vdfold(tab. output) 

Table.f tab; 

int output; 

•endif 

{ 

rag ulong 
rag uchar 
reg Match__t 
reg uchar 
rag int 
rag int 
reg uchar 
rag Match_t 

( (output) 



xey/ n; 

*», *sm, *ends, *aa, "haada; 
*m, *liat, *eurm, *beatm; 
•add, *andfoXd; 

head, len, n_arc ■ tab->n_arc; 
size - tab->eize; 
*arc - tab->src, *tar - tab->tar; 
*base - tab->bass, "table ■ tab- > table; 



) 

else 

t 



) 



if <tab->n_«rc < H_MIN) 

return 0; 
endfold • (■ • arc) + tab->n_«rc; 
curxn ~ base ; 



endfold - (a - t«x) + tab->n_tar; 

cuxm ■ baae*n_arc; 

tf <t*b->n_tar < m__mimj 

roturn vdputinat< tab, a , endfold. Nil (Katch_t* ) . 0 j 



add - MIL (uchar*); 
beiui - >JIL(Match_t*) ; 
len - H^IJJ-l; 
HINIT(key,3,n) ; 
for<; ;) 

{ for(;;) /* ••arch for the longest match •/ 

{ if (Mm - table [key tosize ] ) ) 

goto endsearch; 
liat - m - m->next; /* head of liat 

if<beatm) /• skip over past element* •/ 
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179: 
180 : 
181: 
182: 
18 3: 
184: 
18 5: 
18 6 : 
187 : 
188: 
189: 
190 j 
191: 
192: 
193: 

194 : 

195: 
196: 
197 : 
198: 
199: 
200: 
201: 
202: 
203; 
204: 
205: 
206: 
207: 
208: 
209 : 
210: 
211: 
212: 
) ) 

213 : 

214 : 
215: 
216: 

217: 

218: 

219: 
220: 
221: 
222: 
223: 
224; 
225: 
226: 
227: 
228: 
229 : 
230: 
231: 
232: 

233; 
234: 

235: 
236: 
237: 



{ for{;;) 

( ifja >» beatm+len) 

break; 

if((nt - m->next) liar) 
goro endaearch; 



head - len - (M_MIM-1); /* header before the raatcr */ 
heade - a-rhead; 

for ( ;;, 



i£;{n - in-base ; < n.arc; 



{ 



) 

else 

( 



if(n < head) 

goto next; 
am ■ arc » n! 



if<<n n_are) < head) 

goto next; 
am * tar - n; 



) 



/* make sure that the M_MIN by tea matc.n -/ 
if ( i EQUAL ( heade , am) ) 
goto next; 

/* make sure this is a real match */ 
for (am head, as - a§ < heade; ) 

if (•skit* i- *ea++) 

goto next; 

aa +- M^MIN; 
am +- M_MIK; 
enda • andfold; 

if<{m-baae) < n_arc fi& (n • ( arct n_3rc ; - 3r. < ;er.ds-s 

ends - a*n; 
fbr(; aa < enda; +-*ee, — am) 
if(*»m l» 

goto extend; 

goto extend; 

nexti if((m • m->nextj — list ) 
goto endaearch; 

) 

extend: beatm - m-head; 
n • len; 
len - sa-s; 

if (aa >- endfold) /* already match everything */ 

goto endaearch; 

/* oheex for a -longer match */ 
as H-MIM-1; 
if; len n+1) 

HHEXT(key,sf f n); 
else HINIT{ key f ■« • :i) ; 



endaearch : 

if (beatm) 
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238 

299 

240 

241 

242 

24 3 

244 

245 

246 

247 

246 

249 

250 

251 

252 

2S3 

2S4 

255 

256 

257 

258 

259 

260 • 

261: 

262: 

263 

264 

265 

266 

267 

268 

269 

270 

271 

272 

273 

274 

27 S 

276 

277 

278 

279 

280; 

281 

282 

283 

284 

285 

286 

287 

286 

289 

290 

291 

292 

293 

294 

295 

296 

297 



) 



jr^-" 1 ^ 2QQ att eoitii/n/arvphor./q?/kT?v/SQf^v^rg/^re/ . -.s.'vd' 

( if (output tt vdputinsti tab, add. 9 . beat:^, len) < Oj 

rat urn - 1 ; 

/* add a aufficiant number of suffices */ 

• nda - (3 len) ; 

so - enda - (M_KIN-1) = 

if ( : output) 

curtn » base - (ss-sr:) ; 
else curm ■ base - rv^src - ; es - ;ar ; ; 

Ion - HJfXS-l; 
add - >fIL (uchar*) ; 
beat*! - HIL(Match_t* ) ; 



• lao 

( 



if ( l add) 

as - s; 
9 nda - (« 



add 



/* add om prefix */ 



if (enda > (endfold * (M_MIN-1)) ) 

enda - endfold - (H-J4IM-1); 

if(ai < enda) for<;.j /* add pref ices/suf f ices •/ 



J 



n - key&size: 

if ( ! (n - tableCn] ) ) 

curm->next. = curm; 

else 

( curm- >next - m->naxt; 

ra->next - cum; 

) 

table fn] - cuitot+j 

if({aa +- 1) >- ends) 

bra ft* ; 
HNEXTOcay , aa, n) ; 



if (a > andfold-H-HIH) 
( if(*add) 

add - a; 

break; 

) 

HNEXT()cey,a,n) ; 



/* too ahort. to match •/ 



if (output) /• flush output */ 

return vdputinat { tab, add, android, NlL<Match_f ; , 0 ) ; 
return 0; 



• if _STO_C 

long vddelta(Void_t* arc, long n_src, Void_t* tar, long o_tar, vdd.sc., 
■ else 

long vddeltaterc, a_src, tajr, o_tar, disc) 

voidUt* arc: /• source string if not NULL */ 

long n_arc; /• length of source data ♦/ 



• disc) 
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298: 
299: 
300: 
3 01: 
2: 

304: 

305: 
306: 
307: 
309: 
309: 
310: 
311: 
312: 
313: 
314: 
315: 
316: 
317: 
318: 
319: 
320: 
321: 

322: 

323: 
324: 
325: 
326: 
327: 
328: 
329: 
330: 
331: 
332: 
333 : 
334: 
335: 
336: 
337 : 
338: 

339: 

340: 
341: 
342: 
343: 
344 : 
345: 

346 : 

347 : 

348 : 

349 : 
350: 
331: 
352: 
353 . 
354: 
355 : 
356: 
357: 



Void.t* 

long 
vddisc_ 
•endif 
{ 

reg ir.r 
reg long 
Tablet 



tar; /* target atrir.g if not nuli 
n_tar; /* length of target data 
da.se; /* 10 discipline */ 



size, k, n, window; 
P; 

tab; 



if(:dlsc ii !disc->vritef 'i 

( idisoreadf 44 ((n_tar > 0 44 itar) j| (n_orc > 0 «4 isrc) ) > ) 
return -1; 

if(n_tar < 0) 

return -1; 
if(n_src < 0) 

rusrc ■ 0; 

tab.n_.arc - tab.n_tar - tab. size * 0; 
tab. tar ■ tab . sre - NIL ( uchar* ) ; 
tab base - NIL (Katch_t* ) ; 
tab. table - HIL(Match_t** ) ; 
winit( 4tab. io,diac) ; 

if < disowindow <- 0) 

window - DFLTWINDOW; 
else if (diflowindow > MAXWINDOW) 

window » MAXWIMDOW; 
else window • ( int )diee->windowf 
if ( (long) window > ojtar) 

window - (int)n_tar; 
if(n_arc > 0 at (long)window > n_src) 

window - (int)n_arc; 

/* try to allocate working apace V 
while < window > 0 ) 

{ /* space for the target atrir.g */ 

size • (O-tar ••Oil tar) ? o : window; 
if ( (longjaize > n_tar) 

a lie • <int)n.tar; 
if (9l2e > 0 44 i (tab. tar * (uchar* )malIoc< aize»aizeof (uchar ) ) ) : 
goto reduce .window ? 

/* apace for sliding header or aouree atring */ 
if<n_aro <- 0) /• compreeeion only •/ 

size • tar ? 0 : HEADER(window) : 
elae /* differencing */ 

{ sixe - aro ? 0 : window: 

if ( (long)aize > rt_»rc) 

Size * (int)n_src; 

} 

if (else > 0 44 i (tab. aro * ^uchar* )malloc(eize*eixeof ( ucnar j j j ) 
goto reduee.vlndov; 

/• space for the haeh table alemanta V 
size • window < n__tar ? window : <int)n_tar; 
if(n_arc <- 0j 

size +- window < n.tar ? HEADER {window) : 0; 
else size +- window < n_src ? window : (mt)n_arc; 
if ( i ( tab. base * (Match_t* ) malioc<size*sizeof <Match_c ) ) ) > 
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358 

359 

360 

361 

362 

363 

364 

36S 

366 

367 

368 

369 

370 

371 

372 

373 

374 

375 

376 

377 

373 

379 

360 

381 

382 

383: 

384 

385 

386 

387 

388 

389 

390 

391 

392 

393 

394 

393 

396 

397 

398 

399 

400 

401 

402 

403 

404 

405 

406 

407 

408 

409 

410 

411 

412 

413 

414 

415 

416 

417 



goto reduce _ w i n do w ; 

/* space for the hash table */ 
n • aize/2; 

dc («iz« - n> ; while ((n &» n-lj !- 0); 
if (size < 64) 

size - 64; 

while ( i ( tab. table - (Match.t** )rnalloc( size*siz«of (Match_t* ; )) > 
if((oize »- i) <- 0) 

goto reduce. window; 

/* if get here, successful •/ 

rab.size ■ aize-l; 

break; 

rsduce_vindow : 

if (tab. tar) 

{ freet (Void_t*> tab. tar) ; 

tab. tar - NIL< uchax- ) ; 

1 

if (tab. arc) 

{ f re« < ( void_t • ) tab . arc ) ; 

tab.sro - NlL(uehar» ) ; 

) 

if (tab. bass) 

{ free< (Void_t«) tab. base) ; 

tab. base - NlL(Match_t* ) ; 

) 

if ({window »« 1) <■ 0) 
return -1; 



/» amount proeaaaad */ 
n • 0; 

/• output magic bytea and sizes */ 

for(K - 0; VO_MACIC[k]; k++) 

if CC»_vdvrite)<*tab.io,<uchar*)VD_MACIC,]c) i- k II 
( *_vdputu> ( A tab. io, (uleng)n_tar) <- 0 1 1 
( *_Vdputu> (ttab. io, (ulong)n_sre) <» 0 1 1 
( *_Vdputu) (fitab.io, <ulong) window) <- 0 ) 
goto done; 

/* do one window at a time •/ 
while (n < n^tar) 

{ /* prepare the source string */ 

if (n — are <• 0) /* data compression */ 
( if (n <- 0 ) 

tab. n_aro - 0 : 



else 

{ 



0 1ZS « HEADER (WLndow) ; 
if (tar) 

tab.src - tab. tar > tab.n_tar • siza; 
else memcpy( (Void_t* ) tab. sre, 

<Void_t*) ( tab. tar + tab.r^-ar - size 
aizsi } ; 
tab.n_src - size; 
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418: 
419: 
420 : 
421: 
422: 
423: 
4 24 : 
425: 
426: 
427 : 
426: 
429: 
430: 
431: 
432: 
433: 
434 : 
435: 
436: 
437: 
438: 
439: 
440 ; 
441: 
442: 
) : 

443: 
444 : 
445: 
446: 
447 : 
448: 
449: 
450: 

451: 
452: 
453: 
454 : 
455: 
456: 
4 57 : 
458: 
459: 
460 : 
461: 
462: 
463: 
464: 
465: 
466: 
467: 
466 : 
469: 
470: 
471: 



> 



peregrine zoc . arr cami /n/arvohon/c? /Tcav/so^tware /irc/I ^ :fa /vde • -a/-:dc* 1- * ^ e 



else 

[ 



/* data differencing */ 
iffn < n_arc) 

{ if(n*window > n_arc) 

p 3 n_src- window; 
else p = n; 
if (arc) 

tab.src • (uchar*)arc «* p; 

el»e 

{ size * ( •disc->reacf ; (VD.SOUftCE, tao . 3Z 

window, p, cisc); 

if{Si2e t- wir.dow) 
gcto done ; 

) 

} /* else uae iaat window */ 



} 



cab . o_irc 



window; 



/* prepare the target string */ 

size - (n_tar-n) < window ? ( int ) <n_tar- n) : window; 
tab.n_tar • size; 
if i tar) 

cab.iar - (uchar*)tar +• n; 

else 

{ siza - (*diac->readf > (VD_TAR<*ET, tab. tar, «ize, {long) 



n, due 



) 



if < (long) eize i- tab. n_ tar) 
goto done; 



/* reinitialize table before proceasmg 
for(Jc - tab.aiza; 1c >- 0; - - k ) 

tab.table(k] - NIL(Match_t* ) ; 
K_INIT< tab. quick, tab. recant, tab. rhere) ; 

if (tab.:v_are > 0 at vdf ©ld( fctab, 0 ) < 0) 

goto done; 
if < vdfold( ttab, 1> < 0) 

goto dona; 

n size; 



done : 

if(itar &* tab. tar) 

fraa(<Void_t*)tab.tar) ; 
if {tab. are t& ((n_irc <- 0 *& itar) 

fra«< (VoidLt* ) tab. arc) ; 
if ; tab.baae > 

f raa ( ( Void_t» ) tab . baaa ) ; 
if (tab. table) 

free( (Void.t*) tab. table) ; 

return n; 



(n_src > 0 && i arc) ) ) 
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10 
11 
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43 
44 
45 
46 

47 
48 
49 
50 
51 
52 
53 
54 
55 
56 
57 
59 
59 



1 mr-T^' ~* Taa fttt eQm ' /nygryphor./?VVav/Sof tv/«re /arc/l:.b/yde lta/v^ 
•include -vdelhdr .h" 



/ 



Apply the transformation source - >target to reconstruct rarest 
This coda ia designed to work even if the local nacr.^.ie has 
word size smaller than that of the niach-ne where the delta 
vds computed. A requirement that "long"" on' the local 
-achine must he large enough to hold source and target size 3 
:t la also assumed that if an array is giver., the size ot 
that array In bytes .ttuat be storabie in an 'int". Th-s i-s 
used in various oast from "long" to "lnt rt . 

Vo, kpvSresearch. att. ccm, 5/20/94 



13 : 


*• Written by (Kiem-) Phong 




14 : 


*/ 






15: 


typedef struct . 


.tablets 




16 : 


{ Vdio_t 


io ; 


/• 


17 : 


uchar* 


arc; 


/• 


18: 


long 


n_sre ; 




19: 


uchar* 


tar; 


/* 


20: 


long 


n_tar ; 




21: 


long 


s -org ; 


/* 


22: 


long 


t_orgj 


/* 


23: 


uchar 


data C 1281 ; 


/* 


24: 


char 


s_alloc; 


/* 


25: 


char 


t_alloc; 


/• 


26: 


char 


compress; 


/* 


27: 


X_UDECL< quick 


recent, rhere) ; 


/* 


28. 


> Table_t; 






29 ! 








30: 


•if STD_C 






31: 


static vduafold<TabIe_t* tab) 




32 


• else 






33: 


static vdunfold( tab) 




34 


Table.t* tab; 






35 


•endif 






36 


C 






37 


reg long 


ql2s, copy; 




36 


reg int 


inst, k_type, n 


, r 


39 


reg uchar 


•tax, *src t *to 


• 


40 


reg long 


t , c_addr , n__tar , 


41 


reg Vdio_f 


readf, write* ; 




42 


reg vddisc_t* 


disc; 





ia structure 
source string 

target string 



•/ 



start of window m source 
start of window m target 
buffer for data transferring 
1 if source was allocated 
1 if target was allocated 
1 if compressing only 
address caches 



•/ 

*/ 
•/ 
*/ 

V 
V 



fr : 

n^arc ; 



n_tar - tab->n_tar; 
tar • tab->tar; 
n_src - tab->n_arc; 
arc - tab->»re; 

disc - tab->io . disc; 
readf • disoreadf; 
writef " diac->writef ; 

for(t - 0, c__addr ■ n_src: t < n_tar; ; 
( if((inat - VDCETC( ( vdio_f > tab', ) < 0) 

return -1; 
fc-type - K_CBT{inst); 

if ( i vt>_i9C0PY < Jc.type ) ) 

( if (K_ISMERGE(k_type) ) /• merge/add instruction 



55 



SOOCia <EP 0717353A2J_> 



27 



EP0717353A2 



Sun Aug 14 11:55:45 1994 



10 



15 



20 



25 



30 



35 



40 



45 



50 



60: 
61: 
62; 

63 : 
64: 
65: 
66: 
67: 
68: 
69: 
70: 
71: 
72: 
73: 
74: 
75: 

76 : 

77 : 
78: 
79: 
80: 

> 

81: 
82: 
83: 
84: 

85: 
86: 
87: 
88: 
89: 
90: 
91: 
92: 
93: 
94 : 
95: 
96: 

97 : 

98 : 
99: 

100 : 
101: 
102: 
103: 
104 : 
105: 

106 : 

107 : 

108 : 

109 : 

110 : 
111: 
113: 
113: 
114: 
115: 

116; 

117: 
118: 



I pgrgfl^^a .'zoo . att.eQm! /n/grvphor./p7 /*PV/Sof tvar<> /are/lib / ygal - a/vdusiate - 

aize - A_TCET( mat) : 
elae if (A_ISHERE(inst; ) /* locally coded ADD size •/ 

size ° a_lgeT( ir.at) ; 
else / * r.c n - local ADD size •/ 

{ If ( (size - VDGZ7C( <Vdio_t*) tab) ) < 0) 

return -1; 
if (size >• l_KCRE it 

(Size - (long) (*_vdgetu) ( (Vdio_t*) tab. size; ) 
return -1; 
size • A_GET(olze); 



} 

else 

( 



if((t+slZe> > n.tar) 

return -i; 
c_addr size; 



/* our. of sync "/ 



/* copy data from the delta stream to target */ 

fori ; ; ) 

f if < I tar) 



{ 



else 

i 



) 



if ( (long) (n - aizeof ( tab- >data ) ) size) 

n - ( mt jeize; 
if ( ('^vdread) ( (Vdio_t* )tab, tab->data, n) 

return -1; 
r - ("writef } < VD^TARGET, 

(Void_t* ) tab->data, n, 

tab->t_org+t, diac) ; 
if(r :• n) 

return -1; 



n * { int) size ; 

if ( ( *„Vdread) ( <vdio__t* )tab,tar^t,n) t * 
return -1; 



t * m n; 

if ( (size n> <- 0) 
break; 



) 



i f ( K_I SMERGE ( k_type ) ) 

{ aize • C_TGET< inst) ; 

)c_type - - K_JfERGE; 

goto do .copy; 

) 



if <C_ISH£RE(inat) ) /* locally coded COPY size */ 

size - C_lGET(inst) ; 



elae 

t 



if ((size • VDGETC( { vdio_ t* ) tab ) ) < 0) 

return -1: 
if (size >- I_W0RE »* 

(size - (long) (*_Vdgetu) ( (Vdio^t* )tab. 3i2ej 

return -1; 
else - c_CET(size); 



} < 3) 



do_copy = 



if [ ( t+aize) > n.tar) 
return -1; 



/■ out of sync •/ 
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SO 



119: 
120: 
121: 
122: 

123: 
134: 
125: 
126: 
127: 
i 

128 
129 
130 
131 
132 
133 
134 
135 
136 
13? 
136 
139 
140 
141 
14 2 
143 
144 
145 
146 
147 

n, 

148 : 

149 : 

150 : 
151: 

152 : 

153 ; 

154 : 
155: 

156 : 

157 . 

158: 

159 : 

160 : 
151: 
162: 

163 : 

164 i 

165 : 

166 : 

167 ; 

168 : 

169 : 

170 : 
171: 

172: 

173 : 

174 : 

175 : 



sera ?r:.ne,. ion . fttt,ccwi /r./gryDhon/a7/>?v/S of-vare/9r::/l^b/vc>: Ea/vriu = d*r ^ - 

if ((copy • vdgetc( (Vdio_t*jtab) < 0) 
return -1; 

If (X_type >- K_CVICK && ic.type < ; X^JUieXf K_QTY?E : ; 

copy - tab" >quiclc( copy t ; ; <_ty?e*K_0U:cx <<V2_BI7S ] ; 



else 



if; copy >- I_:iORE 6fc 

(copy - (long) (*_vdgetu: ( (Vdlo_t*; tab. copy) j < : ; 
return -l; 

if(k_type >- K_3ECENT *i x_type < > K_RECENT-X_a.TY?E; 



copy tab->recant;k_typ« 

elae if ( :<_cype — K_HERE) 

copy ° c_addr - ccpy ; 

/• else '<_type — K_SELF •/ 



K_aFDATE(ta5->quicJc, cab->recent , tab->rhere , copy ) ; 
c.addr size : 

If (copy < n_arc) /* copy from aouxca data • 

{ if ( ( copy^»ize) > n_srcj /• oar of syr.c */ 

return -1; 

if(src) 



( 



} 

•lee 

{ 



r. - ; int ) size; 
fr - src-copy; 
if (tar ) 



else 



) 

C r- 



to - tar-t; 
MEMCPY<to,fr.n); 



r • ( *vri*ef ) ( vd_target , void.-- 

disc; ; 



if<r 



tab->t_org-t , 

n> 

return -l; 



1 f ( tab - >compreae ) 



( 
> 

else 

( 

> 

f or ( ; 
( 



copy 
inat 



copy 
inat 



■ tab->t_org 
VD_TAJIGET : 



• tab->e_org; 
VD_SOaRCE; 



tab- >n_arc : 



if i tar) 
( 



elsa 

( 



n - ; int > size ; 

r - t *readf )< lr.st 

(Void.t' ; \ ur". ; 

copy, diacj ; 



n = aizao? t tab- >caca ) : 
if; (long) a > size 

n « j :at ) size : 
r - ( *readf ) ; lr.st 

{ Vcid_t» : iab- >caca . t.. 
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176 
177 
I?8 
1?9 
180 

161: 
182: 
183: 
184: 
185: 
186: 
187: 
168: 
189: 
190: 
191: 

19 2: 
193: 
194 : 
/ 

195 
196 
197 
198 
199 
200 
201 

202 
203 
204 

20 5 
20 6 
207 
206 
209 
210 
211 
212 
313 
214 
215 
216 
217 
218 
219 
220 
321 
222 
223 
224 
225 
226 
227 
228 
229 
230 
231 
232 



pprftyn.r.a 2 = 3 ar.- . earn! /r./grypfrftn/a? /Vov/Sflf tware/arc/llb/Ydel--a/vdusdate . 

copy , diac ;■ ; 
if(r 1= n) 

return -l: 
r « ( *vrxtef ] (VD_TARG£T, 

; Voxd^t* > tab->data « n 



• la 

[ 



tab->t_org-^, disc); 

> 

tf(r 1- n> 

return -1; 

t — n: 

if; (size n; < s 0 ) 

break ; 
oopy n; 

} 

) 

/* eopy from target data */ 
copy n_*rc; 

if (copy >- t I I (copyaize) > n_tar) /* out -of -sync • 
return -1; 

for(;;) /* allow for copying overlapped data */ 
{ reg long e, a; 

if ((a • t-copy) > ai2e) 
s • aize; 

if(tar) 

( to - tax-s-t; fr - tar+copy, n » (int)a 

MEMCPY(to,fr,n); 
t +- n; 
goto next; 

1 

/* hard read/write */ 

a • copy; 

for(;;) 

{ if < (long) (n - ai2eof ( tab->data) ) > 3) 

n - (iot)s; 
r - ( *readf ) {VDJTARGET, 

(VoidUt* ) tab->data, n. 
a + tab->t_org / disc ) : 

if<r 1* r) 

return -l; 
r - ( *vritef ) (VDJTARGET, 

(Void_ t* )tab->data, n, 
t + tab->t_org, diac } ■ 

if(r t- n) 

return *1; 

t n ; 

if ( (a -« n) <• 0) 
break; 

a *• n; 

) 

next: if(<aize s) 0) 
break ; 
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23 3 : 
234: 
235: 
236 : 
237: 
238: 
23S : 
240 ; 
241: 
242: 
243: 
244: 
245: 
246: 
247: 
246: 
249: 
25C: 

2S1: 

252: 

253 : 

254 : 
255: 
256: 
257: 
258: 
259: 
260: 
261: 
262: 
263 : 
264: 
265: 
266 : 
267: 
268: 
269 : 
270: 
271: 

272: 
273: 
274: 
275: 

276 : 

277 : 
278: 
279: 

290: 
281: 
282 : 
281 : 
284 : 
285: 
286 : 
287: 
288: 
289: 
290 : 
291: 
292: 



re v-m 0; 

) 

*if srr.c 

long vdupdate ( void_t* arc, long n_arc , Vcid_t* tar, long n_rar. 
*elae 

long vdupdate ( 9rc, n_src, tar, jx.t*r, disc) 
Void_t* arc; /* source string if any */ 

long n_arc; /* length of arc */ 

vold_t» tar; /* target space if any */ 

long n_tar; /• size of tar */ 

Vddiac.t* disc: 
•end if 
( 

Table_t tab; 

uchar »data, raagic[8J; 
int n, r; 

long t, p. window; 
vdio.f readf, vritef; 



vcdisc_t * 



(itar &t idiaowritef ) ) 



if(!dise || idisc->readf 

return -1; 
readf ■ diac->readf: 
wntef • disc->writef ; 

/* initialize I/O buffer */ 
RINIT{ itab.io,disc) ; 

/* check magic header •/ 
data - (ucher* )<VD_J4XGIC) ; 
for(n * 0; datafn]; --n) 

if { (*_vdread) (4tab. io, magic, n) i- nj 

return -1; 
for(n 1; n >• 0; - - n ) 

if<data(n] i« magic[n] ) 
return -1; 

/* get true target size •/ 

if((t - (long) (*_Vdgetu) (fctab.io,0) ) < 0 M (tar && n_tar »- t) ; 

return - 1 ; 
iV_tar - t; 

/• get true source slse */ 

if((t ■ < long) (•_Vdgetu) ( fctab. io,0 ) ) < 0 I I (src n_aro i- t) i 

return -1; 
rusrc - t; 

/* g«t window alxe */ 

if ((window - ( long) (*_Vdgatu) ( stab. io, 0) ) < 0) 
return -1; 

tab. compress - n_erc 0 ? 1 : 0: 

/• if we have space/ it'll be faster to unfold */ 
tab. tar - tab.src - NIL ( uchar* ) ; 
tab.t_elloc * tab.e_alloc - 0; 
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293: 
294: 
295: 
296: 
297: 
296: 
299 : 
300: 
301: 
) 

302: 
303 : 
304: 
30S: 
306: 
307: 
308: 
309: 
310: 
311: 
312 i 
313; 
314: 
315: 
316: 
317: 
318: 
319: 
330: 
321; 
322: 
323: 
324: 
,n); 
325: 
326: 

327: 
ac) j 
328: 
329: 

330: 

331: 
332: 
333: 
334: 



337: 

338: 

339: 

34 0: 

34] 

342: 

343: 

344: 

345: 

346: 

347; 

348: 

349: 



sftr*orir.« 200 . att . coml /n/qrvphan/c? /Vpv/i Software /«re /lib 'vdupdafr.w , fl 

if (n_tar > 0 &6 ; tar 46 window < ( long ) MAXINT) 

n - t int; window ; 
als* n - 0; 

if(n > 0 66 (tab. tar - (uehar* )malioc< n*si2©of ; achar ; j ) > 
sab.t_alloc - 1; 

if (n.erc > 3 is ! arc 66 window < ( long ;HAXINT) 
n - ( ir.t) wir.dc .-; 

else tf cn_src «• 0 66 window < n_tar 66 itar 66 HEADER (window) < ( long )MAX INT 

n - ( inr) HEADER (window ) ; 
else n - 0; 

if in > 0 &6 (.tab. arc - (uehar* >raalloe( n» 9 izeof ( uehar ;) ) ) 
tab.s_alloc - 1; 

*or(t - 0; z < n.tar; > 

{ tab.t_org - t; /* current location in target atream •/ 

if(n__arc — 0) /* data compression */ 



tab.s.org * 0; 



> 

elaa 



if (t 

else 

{ 



0) 

tab . n_arc 



0; 



tab.n_arc - HEADER (window) ; 
p - t - tab.n_9rc; 
if (tar) 

tab.src - (uehar*) tar * p; 
•lae if(tab.src) 



n - (int )tab.n_arc; 
if (tab. tar) 



) 

•lae 

{ 



data - tab. tar + tab.n_tar - n: 
mamcpy ( ( voicl_t* ) tab .arc, ( Void_t* )data 



r - ( *r*adf ) (VD_TARGET, tab . src n,?, dt 



if (r 1- n) 

goto done; 



) 

/• data d± tf are ncing */ 
tab.n_arc • window; 
i*(t < n_arc) 

{ if ( (t^window) > n_«rc) 

p • n_arc- window; 
alae p - t; 

tab . o_Qrg * p ; 

if (arc) 

tab.src - (uchar*)src * p; 
•la* if(tab.arc) 
( n - ( mt ) tab . n_src ; 

r - (»raadf ) (VD_30URCE. tab. arc, n,p, disc; : 

if(r 1- n) 

goto don«; 
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10 



15 



20 



25 



350 : 
351: 
352 : 
353: 
354 : 
355: 
356 : 
357: 

358 : 

359 : 
360: 
361: 
362: 
O) ; 
363: 
364 : 
365: 

366 : 

367 : 

368 : 
359 : 
370: 
371; 
372: 
373: 
374 : 
373: 
376: 
377: 



Dftrearir.e.KQ atr com! /r. /gryphon /a? /Vw/9cf ih/vdel -« c ^ 1 -. 



) 



) 

ii(tar) 

tab. tar * (uchar* )tar**-t ; 
tab.n_tar ~ ^dov < ( n_tar-r ) ? window 



iar-t) : 



IT czab, quick, tab.racant, tab. rhere) ; 
<vcVaoXold<itab) < 0) 
goto done; 
if ( i tar 64 tab. tar) 

( p« <*writef ) (VD.TARGET, < Void_t» )tab . tar , ( lr.t; tac . 

if(p i« tab.n_tar) 
goto done; 



t tab. n_tar; 



V.ar, t . cis 



dona : 

if (tab. t_alloc) 

free(<Void_t*jtab.tar) ; 
if (tab. «_alloc) 

f roe ( < Void_t * ) tab . arc ) ; 

rtturn t ; 
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10 



15 



20 



25 



30 



35 



40 



45 



50 



1 
2 
3 
4 
5 
6 
7 
8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

31 

12 

33 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

36 

39 

40 

41 

42 

43 

44 

4S 

46 

47 

46 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 



* include " vdelhdr . h" 



#i* STD_C 

static _vdfilbuf ;reg vdio_t* io) 
valae 

static _vdf ilbuf ( lo) 
reg Vdio_t» -<= : 

«endif 

{ reg int n; 

if((n* <*READF( io ) ) <VD_DELTA, BUF(io) . BUFSIZEf 10) ,HERE(io) ,DISC(XO> ) > > 0) 
{ ENDB(iO) - (NEXT(iO) • BUF(io)) + n; 

HERE(iO) n; 

} 

return n; 

} 

*L1 9XD_C 

static _vdflsbuf (reg vdio_t* io) 
• else 

static .vdilabuf (io) 

reg Vdio_t* 

•endif 

{ reg int n; 

if((n - NEXT(io) - BUF(io)) > 0 &6 

(*WRITEF(io) ) (VDJ81»,BOF(io) , n, HERE< to ) , DISC < 10 ) ) I- n) 
return -1; 

HERE(io) +« n; 
NEXt(lQ) - BUF(iO) ; 
return 8UFSriE(io); 

} 

flf 3TD_C 

static ulong _vdgetu(rag Vdio_t* io, reg ulong v) 
• else 

static ulong _vdgatu ( io , v ) 
rag vdio_t* io; 
reg ulong v; 
•endif 

C rag int cj 

for<v 6* IJiORE-l;;) 

{ l£<(c - VOCETC(io)) < 0) 

ratuxn (ulong) ( -XL) ; 
if ( ! ( C4I_W0M) ) 

return ( <v«I_SHIFT) : c): 
v - (v«I_9HIFT) I (c & (I.MORE-1)); 



) 



• if 3TD.C 

static _vdputu(r*g vdio_t* io, ulong v) 
•aloe 

static _vdputu(io, v) 
reg vdio_t« io ; 

rag ulong v; 
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60 
61 
62 
€3 
64 
65 
66 
67 
66 
69 
70 
71 
72 
73 
74 
75 
76 
77 
78 
79 
80 
81 
82 
83 
84 
85 
86 
87 
88 
89 
90 
91 
92 
93 
94 
95 
96 
97 
98 
99 
100 
101 
L02 
103 
104 
105 
10 6 
10 7 
10 8 
109 
110 
111 
112 
113 
114 
115 
116 
117 

lie 

119 



i peragnnt.ZQQ att . -om-. /n/gw^ygyvpy/aQfr^re/are/l ih/vdelEa/vd-. 



#«ndif 

( 

rag uchar 
uchar 



•a, 'ntxt; 
len; 

ctsizeof (ulong 



-II; 



a » r.ix: - &ct»i2eof;<; 
*« - I_C0DE( v) ; 
while; (v »- I.SHIFT) ) 

•--a - I_C0DE< v) | I^MORE; 
len - (next- 8) - 1; 

(REMAIN(lo) < len &6 _vdf labuf \ io ) <» 0) 
return -1; 

next » lo->next; 
Bvitch< len) 

( 

default: mamcpy ( ( Void.t* ) next , < Void_f )a , len) ; next +- len; break; 
case 3 
case 2 
case 1 

} 

io->next 



•next*"* - *«t-; 
•n«xtt» • 



return lan; 

) 

• if STD_C 

static _vdread(Vdio_t* io, rag uchar* a, reg int n) 
»el9e 

static _vdread(io, a, n) 
Vdio_t* io; 
rag uchar* a; 
rag xnt n.- 
*endi* 
( 



rag uchar* 
reg int 



next; 
r , m; 



for<m - n; m > 0; ) 

( if((r - REKAiN<io>) <- 0 4a <r - _vdf ilbuf < io ) ) <- 0) 

break j 
if(r > in) 

r - m; 

next - io->next.; 
KQiCFx* ( fl , next , r ) ; 
io->next - next; 



r ; 



raturn n-m; 



) 

•if _3TD_C 

atatio _vdvrite { Vdio_t* to. reg uchar* a, rag int r.) 
valae 

atatic _vdwrlte(io, %, n) 
vdio z* io; 
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120 
121 
122 
123 
124 
125 
126 
127 

12a 

129 
130 
131 

13 2 
133 
134 
135 
136 
137 
13a 
139 

14 0 
141 
142 
14 3 
144 
145 
146 
147 
148 t 
149: 
150: 



reg uchar* 
reg inr 
••ndif 

( 

reg uchar* 
reg int 



^r«i ? ri"i» ?QQ att mm! /n /pryphori/a? /kpv/sof fwftrft/arc/l-b/vdclra/vdio e 

5; 

n; 



next; 



for<m - n: a > 0: ) 

{ l£((W - 3lBlAlN(io>> <- 0 fi.fi. (w 

break: 
if(w > 31) 

w - m; 

next " io - > next ; 
KEMC?*(next,e, W) ; 
io->next * next; 



return n-m; 



_vdf Isbuf ( lo) ) <■ 0) 



vdbu*io_t jvdbufia 
{ _vdfilbuf, 

.vdflabuf , 

_vdgetu , 

_vdputu , 

_vdread, 

_vdwrito 

}; 
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Claims 

35 

1 . In a computer, the improvement comprising: 

a) program means for producing instructions which allow replication of a second version of a binary file, based 
on a first version, without access to the second version. 

40 

2. A method of replicating a second version of a binary file, which is derived from a first version, comprising the following 
steps: 

a) creating a sequence of instructions, each of which generates a character sequence, which, when concate- 
45 nated, replicate the second version; and 

b) executing the instructions. 

3. For a digital computer, the improvement comprising: 
so a) first program means for 

i) examining a first and a second file and 

ii) producing a sequence of instructions which, when executed, concatenate 

55 A) parts of the first file with 

B) parts of the second file, without access to the second file, 

in order to replicate the second file. 
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4. The improvement according to claim 3. and further comprising second program means tor executing the sequence 
of instructions, for replicating the second file 

5. A method of saving a later version of a computer file, comprising the following steps: 

a) comparing the later version with an earlier version, and identifying 

i) similar passages, where the later version is similar to the earlier version, and 

ii) difference passages, where the later version is different from the earlier version; 

b) storing the address where each similar passage occurs in the earlier version; and 

c) storing each difference passage, together with the address of its occurrence in the later version. 

6. A process of information replication, comprising the following steps: 

a) finding, at a first site, 

i) similarities and 

ii) differences 

between a first file and a second file; 

b) maintaining, at a second site, a copy of the first file; 

c) transmitting the following to the second site: 

25 i) locations of the similarities. 

ii) the differences themselves, and 

iii) information to enable replication of the second file, based on (c)(i). (c)(ii). and the second file. 



75 
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30 



A method of updating an old version of a file into a new version, comprising the following steps: 
a) receiving instructions which order one or more of the following to occur: 



i) copying a specified part of the old version into a specified location of the new version; 

ii) copying a specified part of the new version into a specified location of the new version; and 
35 iii) adding specified bytes to a specified location in the new version; 

and 

b) executing the instructions. 

40 8. A system for generating data indicative of differences between first and second versions of a file, comprising: 
a) means for deriving a sequence comprising the following instructions: 

i) a COPY instruction which orders that characters be copied from 

45 

A) a source location in either the first or second version, to 

B) a destination location in the second version; and 

ii) an ADD instruction which orders that specified characters be added to specified locations in the second 
so version. 
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WORD — ••When buying coconuts, make sure that they are crack free and 
POSITION— 1-* 2-* 3 4 5 6 7 8 9 10 11 



Shake them to make sum that they are 
17 18 19 20 21 22 23 24 



have no makt on them 
12 13 14 15 16 

Now hold a coconut in one hand over a sink 
28 29 3031 32 33 34 35 3637 



heavy with water. 
25 26 27 



and hit it around the center with the claw end of a hammer. 
38 39 40 41 42 43 44 45 46 47 48 4950 



VERSION 
1 



VOCABULARY 
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23 


hold 
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24 
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25 
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FIG. 1A 
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Hawaiian all 
When buyingAcocanuts, make sureA3mMfa$ are crack free and 
1 J 2-* 3 4 5 6 7 8 9 10 11 



have no mold on them. 
12 13 14 15 TS 



Shake them to make sure that they are 
t7 ta 19 20 21 22 23 24 



heavy with water. 
25 26 27 



Now hold, a coconut in on» hand over a sink 
28 29 30 31 32 33 34 35 3537 



a bnck 

and hit it around the center iilh^nii_ | |m i ml nt i liiii||||i i 
38 39 40 41 42 43 44 45 46 47 48 4950 



VERSION 
2 



FIG. 1B 
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When buyina I Hawaiian I coconuts, make sure H are crack free and 
1J 2-* 3 4 5 6 7 8 9 10 11 



have no moid an them. 
12 13 14 15 16 



Shake them to make sure that they are 
17 18 19 20 21 22 23 24 



heavy with water. 
25 26 27 



Now hold a coconut in one hand over a sink 
23 29 3031 32 33 34 35 3637 



and hit it around the center with 

38 39 40 41 42 43 44 45 46 



a brick 





VOCABULARY 


1 When 


16 them 


2 buying 


17 Shake 


3 coconuts 


18 to 


4 make 


19 heavy 


5 sure 


20 with 


6 that 


21 water 


7 they 


22 Now 


8 are 


23 hold 


9 crack 


24 a 


10 fn» 


25 in 


11 and 


26 one 


12 have 


27 hand 


13 no 


28 over 


14 mold 


29 sink 


15 an 


30 hit 



31 it 

32 around 

33 the 

34 center 

35 claw 

36 end 

37 of 

38 hammer ^ 
39* Hqwgbqtt 

40 ait 

41 brick 



NEW 

WORDS 



FIG. 1C 
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When buying coconuts, make sure that they ore crack free and 
"P T=r-^3 4 5 6 7 8 9 10 11 



have no mold on them. 
12 13 14 15 16 



Shake them to make sure that they are 
17 18 19 20 21 22 23 24 



heavy with water. 
25 26 27 



Now hold a coconut in one hand over a sink 
28 29 3031 32 33 34 35 3637 



and hit it around the center with the claw end of a hammer. 
38 39 40 41 42 43 44 45 46 47 48 4950 



VERSION 
1 



COPY 2 1 
INSTRUCTION! 



When buying 
2 



POINTER 

FIG. 1D 



When buying coconuts, make sure that they are crack free and 
1-* 2-* 3 4 5 6 7 8 9 10 11 



have no mold on them. 
12 13 14 15 16 



Shake them to make sure that they are 
17 18 19 20 21 22 23 24 



heavy with water. Now hold a coconut in one hand over a sink 
25 26 27 28 29 30 31 32 33 34 35 3637 

and hit it around the center with the claw end of a hammer. 
38 39 40 41 42 43 44 45 46 47 48 49 50 



VERSION 
1 



When buying Hawaiian 
1 2 3 



ADD 1 Hawaiian — 1 
INSTRUCTI0N_2 



POINTER 



FIG. 1E 



41 



3DOC1D: <EP 07173S3A2J_> 



EP 0 717 353 A2 



When buying [coconuts, moke surel that they are crack free and 
1-» 2-» 3 | 4 5 6 7 a 9 10 11 



have no mold on them. Shake them to make sure that they are 
12 13 14 15 16 17 18 19 20 21 22 23 24 

heavy with water. Now hold a coconut in one hand over a sink 
25 26 27 28 29 3031 32 33 34 35 3637 

and hit it around the center with the claw end of a hammer. 
38 39 40 41 42 43 44 45 46 47 43 4950 



COPY 3 3 
INSTRUCTIONS 



VERSION 
1 



1 



When buying Hawaiian coconuts, make sure 
1 2 3 4 5 6 



POINTER 



FIG. IF 



When buying coconuts, make sure that they are crack free and 
1 J 2-* 3 4 5 6 7 8 9 10 11 

have no mold on them. Shake them to make sure that they are 
12 13 14 15 16 17 18 19 2Q 21 22 23 24 



heavy with water. 

25 26 27 



Now hold a coconut in one hand over a sink 
28 29 3031 32 33 34 35 3637 



and hit it around the center with the claw end of a hammer. 
38 39 40 41 42 43 44 45 46 47 48 49 50 



VERSION 

1 



ADO 1 all 



INSTRUCTI0N.4 



When buying Hawaiian coconuts, make sure all 
12 3 4 5 6 7 



POINTER 



FIG. 1G 
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When buying coconuts, make sure that they 
1 J 2-* 3 4 5 6 7 



are crack free- and 
8 9 10 11 



hove no mold on them. 
12 13 14 15 16 



Shake them to make sure that they are 
17 t8 19 2Q 2t 22 23 24 



heavy with water. 
25 26 27 



Now hold a coconut in one hand over a sink 
28 29 3031 32 33 34 35 3637 



and hit it around the center with) the clow end of a hammer. 
38 39 40 41 42 43 44 45 46 47 484950 

POINTER 



VERSION 
1 



\ 



When buying Hawaiian coconuts, make sure all [are crack free and 

1-» 2-* 3 4 5 6 7 18 9 10 11 

. i 

I have no mold on them. Shake them to make sure that they are 

112 13 14 15 16 17 18 19 20 21 22 23 24 

h heavy with water. Now hold a coconut in one hand over a sink 
[25 26 27 28 29 3031 32 33 34 35 3637 

[_gndhit it around the center withj" 
38^S"40 4T ~42~43 44* 



COPY 37 8 
INSTRtOOM_5 



FIG. 1H 
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When buying coconuts, make sure that they are crack free and 
1> 2-± 3 4 5 6 7 8 9 TO Tt 



have no mold on them. 
12 13 14 15 16 



Shake them to make sure that they are 
17 18 19 20 21 22 23 24 



heavy with water. 
25 26 27 



Now hold a coconut in one hand over a sink 
28 29 3031 32 33 34 35 3637 



and hit it around the center with the claw end of a hammer. 
38 39 40 41 42 43 44 45 46 47 48 49 50 



VERSION 
1 



ADD 2 o brick ; 
INSTRUCTION^ 



When buying Hawaiian coconuts, make sure all are crack free and 
1-* 2-i 3 4 5 6 7 8 9 10 11 

have no mold on them. Shake them to make sure that they are 
12 13 14 15 16 17 18 19 20 21 22 23 24 

heavy with water. Now hold a coconut in one hand over a sink 
25 26 27 28 29 30 31 32 33 34 35 3637 



1 



and hit it around the center with a brick 
38 39 40 41 42 45 44 4546 
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POINTER 



FIG. 11 
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FIG. 1 



VERSION 
1 



LENGTH 16 



CHARACTER: abcdabcdabcdefgh 

t t t t 

POSITION: 0 12 15 



VERSION 
2 



LENGTH T7 



CHARACTER: abcdxyxyxyxybcdef 

t t t t t t 

POSITION; 16 17 18 19 20 32 



TRANSFORMATION 



COPY 4 0 
ADO 2 x.y 
COPY 6 20 
COPY 5 9 



FIG. 2A 
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VERSION 
1 



VERSION 
2 



CHARACTER: r [o__ b_c_d] a b c 

| t t t t t 

POSITION: | 0 1 2 3 6 
I , 

, i 



character: 
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instruction(copy 4- 0 
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t t t 
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FIG. 2B 
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POINTER (AT 20) 



FIG. 2C 
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VERSION 
1 



CHARACTER: abcdabcdabcdefgh 

t t t t t t t t 

POSITION: 0 1 2 3 6 9 12 15 



VERSION 
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CHARACTER: abed [x^y] x y x y x y 

t t t V t 
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INSTRUCTION (COPY 6 20 



POINTER (AT 22) 



FIG. 2D 
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VERSION 
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FIG. 2E 
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TRANSFORMATION 



c 


d a 


b 


c d 


a b 


c 


d x 


y 


x y 


x y 


1. 


COPY 




4 


0 


2. 


ADD 




2 


x.y 


3. 


COPY 




6 


20 


4. 


COPY 




5 


9 



FIG. 2 



1. n = length(VERSI0N1); 

2. C * 0 

3. while(tnjft) 

4. { inst = readinstQ; 

5. if(inst == end-of-fde) 

6. return; 

7. s = readsizeO; 

8. ifOnst == ADD) 

9. readdota(VERStON2+c,s); 

10. else 

11. { p = reodposQ; 

12. if(p < n) 

13 copy(VERSI0N2+c,VERSI0N1+p,s); 

H else copy(VERSI0N2+c,VERSION2+p-n,s); 

16. c * c +■ s; 

17. } 

FIG. 3 



010QQ100 0 

00000010 x y 

01000110 20 
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1. Initialize table of positions T to empty; 

2. process(VERSI0N1); 

3. pracess/VERSION2); 

4. process(seq) 



5. | n « length(VERSIONt); 

6. m = length(seq); 

7. c = 0; 

8. add - -1; 

9. pos = -1; 

10. len a-MIN-1; 

11. whilst) 

12. | while(l) 

13. { p = search(T,seq,c,len); 

14. if(p < 0) 

15. break; 

16. len = extend(T.seq,c,p,len); 

17. pos - p; 

18. | 

19. if (pos < 0) 

20. | if(add < 0) 

21. add = c; 

22. if(seq VERSION 1) 

23. insertfT,seq,c.Q): 

24. else ins6rt(T,s6q,cj>}; 

25. c = c + 1; 

26. ( 

27. else 

28! | if(seq == VERSI0N2) 

29. writeinst(add,c,pos,len); 

30. p = c+len-(MIN-1); 

31. whik(p < c+ten) 

32. { iflfseq VSRSONT) 

33. insertjT,seq,p,0V 

34. else insert(T,seq l p 1 n)L 

35. p = p+f 

36. 1 

37. c » c + len; 

38. odd = -1; pos = -1; len = MIN-1; 

39. \ 

40. if(c >- m-MW) 

41. break; 

42. | 

43. if(seq == VERSI0N2 and add >= 0) 

44. writeinst(add,m,-1,0); 

45. } 



FIG. 5 
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1 . insert(T,seq,p,orig) 

2. { key = seq[p,p+1 p+MIN-1]; 

3. pos = p+orig; 

4. Insert (key.pos) into T; 

5. \ 



FIG. 6 



search(T,seq,c,len) 
{ n = length(VERSI0N1); 
p = c + len - (MM-1); 
key = seq(p,p+1,...,p+MIM-1]; 
for(each entry e in T that matches key) 
j pos = position(e); 
if(p >= n) 
{ str = VERSI0N2; 
q = pos-n; 

I str » VERS10W1; 
a, » pas 

d = q - (len - (MIN-1)'); 

if(d >=» 0) 

if(seqfc.c+1 p-1] == str[d,d+1,....q- 

retum pos - (tan - (W*-f) ); 

\ 

return -T; 



FIG. 7 
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1. extend(T,seq,c,p,!en) 

2. | n * length(VERSI0N1); 

3. if(p < n) 

4. str = VERSION t; 

5. else 

6. | str = VERSION2; 

7. p = p-n; 
8- I 

9. m = length(seq); i = c +• len + t; 

10. n = lencth(str); j = p + len + 1; 

11. whfle(i < m and j < n) 

12. if(seq[i] != seq[j]) 

13. break; 

14. return i-c; 

15. \ 



FIG. 8 



VERSION1 | a b cdabcdabcdefgh 

^ A A A A A A A A 

VERSION2 1 a bcdxyxyxyxybcdef 

^ A A A A A AAA 
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01000100 0 
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(54) Efficient and secure update of software and data 



(57) The invention concerns apparatus for updating 
a data file, from an earlier version to a later version. The 
invention compares the earlier version with a later ver- 
sion, and derives a transformation, which contains infor- 
mation as to the similarities and differences. The 
invention then processes the earlier version, using the 
transformation, in order to derive the later version, with- 
out reference to the later version itself. 



The invention allows multiple versions of an original 
file, such as bank records located at multiple locations, 
to be updated by transmitting the transformation to the 
multiple locations, instead of transmitting the updated 
files themselves. The procedure is highly resistant to 
interception of the updated data, because the transfor- 
mation, in general, does not contain the entire contents 
of the later versions. 
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